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PREFACE 


In  1976-1978,  as  part  of  a  study  of  "Air  Force  Health  Delivery  Systems"  for  Project  AIR 
FORCE,  Rand  assisted  the  Air  Force  Surgeon  General  with  a  demonstration  project  in  the  use 
of  physician’s  extenders  in  primary  medicine  outpatient  clinics.  This  report,  one  of  a  series 
presenting  Rand’s  evaluation  of  the  demonstration  project,  assesses  the  quality  of  care  de¬ 
livered  by  the  extenders.  Two  other  reports  are  forthcoming;  the  first  will  provide  an  overview 
of  the  project,  focusing  on  operational  and  economic  issues;  the  second  will  describe  patients’ 
attitudes  toward  physician’s  extenders  and  the  demonstration  project. 

In  addition  to  its  immediate  audience  in  the  office  of  the  Surgeon  General,  the  present 
report  should  be  of  interest  to  the  wider  medical  community,  since  it  documents  the  quality 
of  care  that  can  be  achieved  with  physician’s  extenders  in  a  favorable  institutional  environ¬ 
ment. 

Other  topics  addressed  by  the  health-delivery-systems  project  include  the  supply  of  physi¬ 
cians  and  the  demand  for  outpatient  services.  The  research  is  conducted  for  Project  AIR 
FORCE  under  Rand’s  Manpower,  Personnel,  and  Training  Program. 
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SUMMARY 


With  the  end  of  the  physician  draft,  the  Air  Force  has  experienced  a  decrease  in  the 
number  of  available  physicians,  particularly  for  outpatient  (ambulatory)  care.  In  1976,  Rand 
proposed  alleviating  that  shortage  by  introducing  large  numbers  of  physician’s  extenders 
(physician’s  assistants  and  primary  care  nurse  practitioners)  into  the  Air  Force  setting;  these 
extenders  might  assume  a  sizable  proportion  of  the  patient  load  previously  treated  by  physi¬ 
cians.  Agreeing  with  the  potential  merit  of  Rand’s  recommendations,  the  Air  Force  established 
a  demonstration  project  in  1976  at  four  base  hospitals  to  test  the  feasibility  of  using  physician’s 
extenders  (PEs)  to  provide  a  large  portion  of  primary  care  in  the  clinics.  This  report  presents 
an  evaluation  of  the  quality  of  care  provided  during  the  demonstration  project,  by  comparing 
PEs’  performance  with  that  of  physicians  working  in  the  same  setting. 

We  adduced  several  strands  of  evidence  in  evaluating  the  medical  appropriateness  of 
having  PEs  assume  a  substantial  portion  of  the  workload.  We  assembled  the  results  of  condi¬ 
tion-specific  quality-of-care  criteria  applied  to  the  care  provided;  reviewed  differences  in  the 
pattern  of  ordering  tests,  procedures,  and  return  visits;  and  analyzed  the  supervisory-consulta¬ 
tive  relationships  between  physicians  and  extenders. 

We  find  that  the  Air  Force  can  deliver  the  same  quality  of  care  when  PEs  treat  a  sizable 
percentage  of  patients  formerly  treated  by  physicians.  We  first  compared  PEs’and  physicians’ 
treatment  of  common,  simple,  outpatient  problems  for  five  classes  of  quality-of-care  criteria 
defining  desirable  diagnostic  actions,  desirable  therapeutic  actions,  undesirable  diagnostic 
actions,  undesirable  therapeutic  actions,  and  desirable  disposition  (follow-up)  actions.  We 
found  that  physician’s  assistants  (PAs)  performed  as  well  as  or  better  than  physicians  in  25 
out  of  28  criteria,  and  primary  care  nurse  practitioners  (PCNPs)  performed  as  well  as  or  better 
than  physicians  in  14  out  of  19  criteria.  For  most  criteria,  we  found  no  significant  differences 
among  the  three  groups  of  providers,  nor  could  we  identify  any  systematic  deficiency  on  the 
part  of  any  group  of  providers. 

In  addition,  there  was  no  overall  difference  in  the  performance  of  PAs  and  PCNPs. 
Furthermore,  we  did  not  find  a  decay  in  PAs’  performance  when  1974  and  1977  data 
were  compared,  nor  did  we  identify  differences  between  recently  graduated  and  ear¬ 
lier  graduated  PAs.  Thus,  the  “product”  being  graduated  from  the  Air  Force’s  own,  in- 
house  training  program  appears  to  be  both  consistent  and  stable. 

We  found  that  PEs  ordered  diagnostic  procedures  at  rates  similar  to  those  of  physicians. 
Some  differences  in  ordering  rates  were  observed,  but  there  was  no  consistent  evidence  for 
significant  overburdening  of  the  Air  Force’s  care  delivery  system.  Analysis  of  return  visits, 
referrals,  and  hospital  admissions  provided  further  evidence  of  appropriate  extender  behav¬ 
ior. 

Finally,  supervision  of  the  extenders  by  physicians  occurred  with  reasonable  frequency, 
and  was  more  likely  to  occur  for  cases  judged  as  complex  or  serious.  This  suggests  that 
extenders’  care  was  adequately  supervised.  Consultation  took  up  only  a  modest  amount 
of  the  physicians’  time. 

In  all  these  aspects — quality,  utilization,  consultation — PEs  perform  satisfactorily.  Insofar 
as  we  can  determine,  extenders  measure  up  to  the  performance  level  of  physicians  working 
in  the  same  setting.  Their  performance  in  the  demonstration  project  also  compares  well  with 
previous  studies  of  extender  quality,  even  in  light  of  the  higher  ratio  of  PEs  to  physicians  at 
the  demonstration  bases. 

We  believe  that  the  Air  Force  can  continue  to  employ  PEs  in  outpatient  clinics  without 
diminution  in  the  quality  of  care.  Indeed,  these  findings  medically  justify  this  expanded  use 
of  PEs  in  primary  care  settings  throughout  the  Air  Force. 
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I.  INTRODUCTION 


BACKGROUND 

Since  the  end  of  the  physician  draft,  fewer  physicians  have  been  available  to  the  Air  Force. 
The  shortage  has  been,  and  will  continue  to  be,  particularly  acute  in  outpatient  (ambulatory) 
care,  a  traditional  arena  for  general  medical  officers  (GMOs)  who  are  in  especially  short 
supply. 

In  1976,  Rand  proposed  alleviating  the  physician  shortage  by  introducing  large  numbers 
of  physician’s  extenders  (PEs)  into  the  Air  Force  setting,  where  they  might  assume  a  sizable 
portion  of  the  patient  load  previously  treated  by  physicians.  Although  the  PE  concept  might 
have  merit  under  any  circumstances,  it  appeared  especially  important  because  of  the  shortage. 

Rand  recommended  that  the  outpatient  primary  medicine  areas  be  staffed  by  teams  of 
providers  consisting  of  one  physician  and  two  or  three  highly-trained  PEs  (the  term  includes 
both  physician’s  assistants  (PAs)  and  Primary  Care  Nurse  Practitioners  (PCNPs)).1  In  this  way, 
the  Air  Force  might  deal  with  the  shortage  of  primary  medicine  physicians  while  providing 
high-quality  and  responsive  medical  care  in  a  cost-effective  manner.  Because  the 
recommendations  required  extrapolation  beyond  actual  experience  with  PEs  in  the  Air  Force, 
and  because  the  Air  Force  did  not  have  extenders  to  implement  the  concept  on  a  large  scale, 
Rand  recommended  a  demonstration  program  at  a  small  number  of  installations.  The  Surgeon 
General  approved  this  recommendation. 

In  the  fall  of  1976,  the  Air  Force  began  the  demonstration  project  in  the  primary  medicine 
clinics  of  four  base  hospitals  (Chanute,  Dyess,  Fairchild,  and  Nellis)  to  evaluate  a  care-deliver¬ 
ing  system  using  a  large  number  of  PEs.  The  purpose  of  this  report  (one  in  a  series  concerning 
the  demonstration  project)  is  to  examine  the  quality  of  care  provided  by  PEs  during  the 
demonstration. 

The  report  assesses  the  quality  of  primary  medical  care  when  a  substantial  portion  of  it 
is  delivered  by  PCNPs  and  PAs,  in  concert  with  physicians.  For  this  assessment,  we  use  a 
comparative  method;  that  is,  we  compare  the  PEs’  quality  of  care  with  that  of  physicians,  at 
least  for  the  type  of  routine  outpatient  problems  that  PAs  and  PCNPs  are  trained  to  treat. 

We  present  the  results  of  specific  quality-of-care  criteria  applied  to  the  care  that  PEs 
provide;  review  the  differences  in  the  way  they  order  tests,  procedures,  and  return  visits;  and 
analyze  the  supervisory-consultative  relationships  between  physicians  and  PEs.  Several 
strands  of  evidence  are  thus  available  for  the  comparison. 


THE  DEMONSTRATION  PROJECT 

As  organizational  steps,  the  demonstration  project  proceeded  to: 

•  Enrich  the  mix  of  PEs  in  primary  medicine  clinics,  principally  using  PAs; 

•  Organize  providers  into  teams,  with  a  physician  as  supervisor; 

•  Use  PEs  to  provide  first  echelon  of  care.  Physician  serves  as  a  supervisor,  consultant, 
referral  point,  and  regular  provider  for  previously  seen  complex  patients. 

•  Organize  patients  (including  retiree  families)  into  panels,  with  one  team  treating  one 
panel; 

'The  term  "physician's  extenders"  is  controversial,  and  will  be  discussed  below  together  with  a  description  of  PAs  and 
PCNPs 
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•  Establish  a  system  where  all  patients  except  true  emergencies  are  seen  by  appoint¬ 
ment; 

•  Furnish  (rationed)  direct  access  to  a  physician  for  those  (few)  patients  who  strongly 
prefer  a  physician; 

•  Retain  other  usual  system  features  (personnel  rotation,  facilities,  support  personnel). 

The  most  important  project  feature  was  the  high  ratio  of  PEs  to  primary  care  physacians: 
either  two  or  three  to  one.  This  ratio  was  based  on  a  1974  study,  conducted  at  nine  Air  Force 
installations.  Because  of  their  greater  availability,  the  project  included  more  PAs  than  PCNPs. 

The  providers  were  organized  into  teams  that  replaced  the  traditional  primary  care  clinics 
(general  therapy  and  flight  surgeons).2  In  these  teams,  the  physician  played  a  supervisory  role. 
Most  outpatients  initially  saw  an  extender.  The  physician  was  responsible  for  monitoring 
extenders’  care,  furnishing  consultation,  caring  directly  for  the  more  difficult  patients,  and 
providing  general  medicine  inpatient  care. 

With  the  teams  organized  in  this  fashion,  it  was  possible  to  do  something  about  one  of  the 
persistent  complaints  concerning  military  medicine:  its  impersonal  nature,  typical  of  many 
institutional  systems.  Patients  were  assigned  in  panels  so  that  each  active  and  retired 
household  using  the  base  for  care  had  an  identified  team  of  practitioners. 

Although  the  project  contained  important  elements  of  the  family  practice  approach,  it  was 
not  identical  to  it,  and  in  fact  offered  some  additional  flexibility.  Team  chiefs  were  not  limited 
to  family  practitioners,  although  family  practitioners  and  internists  were  favored.  The  demon¬ 
stration  bases  also  retained  the  traditional  Obstetrics/Gynecology  and  Pediatric  clinics.  The 
division  of  labor  between  the  teams  and  the  Obstetrics/Gynecology  and  Pediatric  clinics 
depended  on  their  relative  richness  of  staffing.  In  practice,  because  of  shortages  of  pe¬ 
diatrics  personnel,  the  teams  cared  for  all  except  young  children. 


PHYSICIAN’S  EXTENDERS 

The  term  "physician’s  extenders”  is  controversial.  Other  authors  prefer  to  use  the  term 
"new  health  professionals,”  or  "allied  health  professionals,”  but  virtually  all  workers  in  this 
field,  including  us,  agree  that  no  single  term  yet  devised  is  entirely  appropriate.  PEs  are 
non-physicians  who  perform  some  of  the  medical  and  administrative  tasks  traditionally  per¬ 
formed  by  physicians.  In  its  broadest  sense,  the  term  "physician’s  extender”  conceivably  could 
refer  to  receptionists  (who  have  often  performed  triage  over  the  telephone)  in  physicians’ 
offices,  medical  corpsmen  or  technicians,  and  nurse  anesthetists,  among  others.  This  report, 
however,  uses  the  term,  "physician’s  extenders”  to  refer  to  only  two  types  of  new  health 
professionals:  physician’s  assistants  and  nurse  practitioners.3 

The  Air  Force  PA  is  a  former  corpsman  who  has  graduated  from  a  two-year  program  of 
instruction  operated  by  the  Air  Force.  The  program  includes  one  year  of  classroom  work  in 
the  basic  sciences,  and  a  one-year  rotation  through  the  outpatient  departments  of  a  large  Air 
Force  hospital.  The  PA  is  therefore  trained  to  diagnose  and  treat  common  illnesses,  and  can 
also  help  manage  complex  patient  problems  under  the  supervision  of  a  physician. 

The  Air  Force  nurse  practitioner  (NP)  is  a  registered  nurse  who  has  taken  a  variable 
amount  of  additional  training,  similar  to  that  received  by  the  PA,  but  usually  less  extensive. 

2Depending  on  the  physician's  training,  they  could  also  absorb  the  workload  of  internal  medicine. 

;,AMOSISTS  (volunteer  corpsmen  who  receive  two  weeks'  classroom  instruction  followed  by  ten  weeks  of  on-the-job 
training,  and  who  then  deliver  care  by  strict  adherence  to  printed  algorithms  as  part  of  the  so-called  Automated  Military 
Outpatient  Systeml  are  also  excluded  from  our  definition  of  the  term  physician’s  extenders,  even  though  they  perform  in 
military  medical  settings  (Vickery  et  al.,  1975). 
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Like  the  PA,  the  NP  is  trained  to  diagnose  and  treat  common  illnesses  and  to  help  manage 
complex  patient  problems  under  a  physician’s  supervision.  Unlike  the  PAs  in  the  Air  Force, 
however,  who  have  functioned  mainly  as  generalists,  NPs  have  often  been  specially  trained 
in  obstetrics/ gynecology  and  pediatrics,  as  well  as  primary  care. 

No  branch  of  the  military  uses  PAs  and  PCNPs  in  precisely  the  same  way,  but  in  the  Air 
Force  they  are  virtually  interchangeable.  Despite  the  differences  in  both  their  background  and 
training,  they  are  expected  to  handle  similar  types  of  patients  with  similar  types  of  diseases 
in  similar  settings.  It  is  reasonable,  then,  to  consider  them  jointly  in  our  analysis,  although  we 
also  examine  their  performance  separately. 

Numerous  studies  examining  selected  aspects  of  performance  in  different  settings  have 
shown  that  PEs  can  provide  a  portion  of  the  care  formerly  provided  by  physicians,  without 
a  sacrifice  in  the  level  of  quality.  The  reader  is  referred  to  the  sweeping  evaluation  by  Appel 
ar.u  Lowin  (1975),  as  well  as  to  the  more  specific  reports  of  Perry  (1977),  Lewis  et  al.  (1969), 
Sackett  et  al.  (1974),  and  Levine  et  al.  (1976).  If  one  wishes  to  look  specifically  at  the  technical 
process  of  care  rendered  by  PEs,  limited  data  are  available  in  studies  by  Komaroff  (1974), 
Tompkins  et  al.  (1977),  Sibley  et  al.  (1975),  and  Duttera  and  Harlan  (1978).  There  has  been  one 
published  study  (Kane,  Olsen,  and  Castle,  1976)  in  which  PEs’  technical  process  of  care  was 
compared  with  that  provided  by  their  physician  preceptors,  through  the  use  of  an  encounter 
form  (described  below)  similar  to  that  used  in  our  work. 

Our  study  reexamines  the  issue  of  the  quality  of  care  that  can  be  provided  by  PEs.  It 
focuses  on  the  technical  process  of  care  in  the  military  outpatient  medical  setting.  The  demon¬ 
stration  project’s  high  ratio  of  extenders  to  physicians  raises  the  question  of  maintenance  of 
quality  under  this  condition.  The  great  absolute  number  of  extenders,  differences  in  their 
training,  and  variations  in  the  amount  of  experience  they  have  had  (possibly  affecting  the 
quality  of  their  current  practice) — these  are  other  important  reasons  for  measuring  quality 
of  care  within  the  context  of  the  demonstration  project. 


OUTLINE  OF  THE  STUDY 

Section  II  of  this  report  presents  the  methods  of  data  collection  and  measurement.  Section 
III  explores  five  categories  of  condition-specific  criteria  that  measure  quality,  and  analyzes  the 
results.  Section  IV  presents  a  utilization  analysis  of  ordering  rates  for  tests  and  procedures. 
It  also  offers  a  brief  analysis  of  the  clinic’s  self-generation  of  visits.  Section  V  examines 
supervision  of  PEs,  a  measure  of  system  quality.  We  examine  the  consultation  and  disposition 
patterns  for  physicians  and  PEs.  Section  VI  presents  our  general  conclusions. 


II.  DATA  AND  METHODS 


QUALITY  OF  CARE  MEASUREMENT 

Quality  of  care  criteria  are  judgmental  statements  regarding  what  is  to  be  considered 
"good  and  bad”  medical  care.  Medical  care  can  be  viewed  as  consisting  of  "technical”  aspects 
(what  is  done  or  not  done  to  the  patient,  and  how  it  is  done)  and  "caring”  aspects  (humaneness 
of  interaction,  establishment  of  rapport  between  the  patient  and  the  provider,  etc.)  (Howell, 
1976;  Goldberg,  1977).  The  data  source,  the  Patient  Contact  Record  (described  below),  dictates 
that  our  quality  of  care  criteria  deal  solely  with  the  technical  aspects  of  care.  We  are  also 
generally  limited  to  evaluation  of  the  "process”  of  care  rather  than  the  "outcome”  of  care 
(Donabedian,  1966).  Process  measures  concentrate  on  the  procedures,  methods,  and  strategies 
of  care,  while  outcome  measures  focus  on  the  effects  of  treatment  (Kane,  1976).  These  topics 
have  been  thoroughly  reviewed  (Brook,  1973;  Brook  and  Williams,  1976;  Williams  and  Brook, 
1978;  Greenfield  et  al.,  1977),  and  will  not  be  discussed  here. 

The  general  limitations  on  evaluating  the  technical  quality  of  care  offer  few  problems  in 
the  general  outpatient  setting,  where  conditions  are  relatively  straightforward  (though  not 
necessarily  well-defined).  In  the  outpatient  setting,  many  presenting  conditions  treated  are 
common  and  conventionally  managed,  and  there  is  a  relatively  clear  connection  between  the 
process  of  care  and  its  outcome  (cure  or  symptom  relief).  Therefore,  we  can  use  such  criteria 
to  evaluate  the  care  rendered  by  physician’s  extenders  (PEs)  and  to  understand  how  that  care 
compares  and  contrasts  with  the  care  rendered  by  physicians  working  in  similar  settings. 

Quality  of  care  criteria  were  selected  to  cover  a  range  of  age  groups;  acute  and  chronic 
diseases;  preventive,  diagnostic,  and  therapeutic  services;  and  physical  and  mental  concerns. 


RELATIVE  QUALITY  OF  CARE 

Quality  of  care  criteria  need  not  be  applied  as  all-or-nothing  (100  percent  or  0  percent) 
compliance  standards.  "Absolute  quality”  values  of  100  percent  or  0  percent  (compliance  with 
a  criterion)  are  unlikely  to  represent  the  best  or  worst  possible  care,  because  patients  always 
vary  enough  to  render  absolute  values  questionable.  For  example,  a  provider  who  decides  to 
check  the  box  "coryza”  (runny  nose)  as  the  diagnosis,  even  though  the  patient  also  complains 
of  a  mild  sore  throat,  may  still  decide  to  swab  the  throat  to  obtain  a  throat  culture.  The  record 
will  then  show  that  throat  cultures  were  obtained  from  some  patients  with  "coryza”,  while  our 
criterion  may  specify  that  no  "coryza”  patients  need  throat  cultures,  but  patients  with 
"pharyngitis”  (sore  throat)  do.  Where  does  that  leave  our  criterion?  We  believe  that  the 
criterion  remains  intact — its  relativity  a  virtue,  not  a  flaw.  Given  two  groups  of  patients  with 
the  diagnoses  just  mentioned,  for  example,  it  is  a  reasonable  criterion  to  say  that  a  substantial¬ 
ly  higher  percentage  of  pharyngitis  patients  should  have  throat  cultures  taken. 

How  high  (or  how  low)  should  percentages  of  compliance  be?  In  view  of  such  problems  as 
multiple  (or  unrecorded)  diagnoses,  variations  in  disease  severity,  and  other  factors,  arbitrary 
numbers  cannot  be  justified.  However,  so  long  as  we  can  state  that  "higher”  is  "better,”  we 
can  still  make  meaningful  statements  about  the  relotiuequality  of  care,  which  is  what  we  want 
to  do.  We  have  the  further  advantage  of  wishing  to  compare  groupsof  providers:  one  physician 
group  with  one  extender  group,  or  one  extender  group  with  another.  Therefore,  we  compare 
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their  performances  according  to  their  compliance  rates  for  each  criterion.1 

The  gist  of  the  comparative  method  of  evaluating  quality  of  care  is  therefore:  two  (or  more) 
groups  to  compare;  a  quality  of  care  criterion;  and  a  compliance  standard  which,  while  not 
absolute,  does  specify  if  better  quality  of  care  is  represented  by  a  statistically  significantly 
higher  or  lower  percentage  performance.  Thus,  we  ameliorate  otherwise  persistent  problems 
of  arguing  whether  70  percent,  85  percent,  or  100  percent  performance  on  a  certain  quality 
of  care  criterion  is  "good”  care  or  not. 

For  some  patients  with  a  particular  condition,  the  failure  to  comply  with  an  applicable 
criterion  can  be  a  perfectly  sound  decision,  not  reflecting  any  decrement  in  the  quality  of  care. 
For  example,  a  blood  sugar  or  urinalysis  need  not  be  ordered  on  every  visit  for  a  diabetic 
patient.  The  validity  of  our  method  depends  on  a  pair  of  assumptions  concerning  how  these 
exceptional  cases  are  treated  by  practioners. 

First,  we  assume  that  the  populations  of  patients  seen  by  each  practitioner  type  are  similar 
in  their  need  for  the  criterion  to  be  followed.  If  exceptional  cases  are  not  equally  distributed 
among  the  practitioner  types,  then  comparisons  of  compliance  rates  would  reflect  these  differ¬ 
ent  distributions  and  not  necessarily  any  real  differences  in  quality.  Patient  populations  may 
also  differ  by  the  severity  of  the  illness.  Some  criteria  are  more  compelling  for  serious  or 
complicated  cases  than  for  simple  cases.  Since  at  most  clinics,  triage  of  the  patients  sent  more 
serious  cases  to  the  physicians  more  often  than  to  the  extenders,  good  quality  would  require 
higher  compliance  by  the  physicians  with  such  criteria. 

The  second  assumption  pertains  to  the  treatment  of  exceptional  cases.  Physicians  and 
extenders  may  differ  in  their  ability  to  recognize  the  cases  that  are  exceptions  to  the  criterion. 
Even  if  such  cases  are  evenly  distributed  among  practitioner  types  (i.e.,  if  the  first  assumption 
holds),  extenders  may  be  more  structured  in  their  decisions,  and  may  more  often  comply  with 
the  criterion  in  those  cases  where  it  is  not  strictly  required.  If  extenders  more  often  follow  the 
criterion  in  cases  where  physicians  would  justifiably  not,  then  compliance  by  extenders  would 
be  increased  without  any  corresponding  increase  in  quality.  The  increase  in  compliance  might 
be  sufficient  to  hide  an  otherwise  poorer  performance  by  the  extender,  or  could  even  lead  to 
the  erroneous  conclusion  that  the  extender  outperformed  the  physician.  In  considering  the 
results  from  the  analysis  of  performance  on  the  criteria  we  have  formulated,  the  likely  fre¬ 
quency  of  exceptions  to  the  criterion  must  be  kept  in  mind.  If  they  are  very  frequent,  the 
effects  of  violations  of  these  two  assumptions  could  be  important. 


THE  PATIENT  CONTACT  RECORD  (ENCOUNTER  FORM)  AS  A  DATA 
COLLECTION  INSTRUMENT 

For  all  the  measures  described  in  following  sections,  data  were  collected  on  a  Patient 
Contact  Record  (App.  A);  the  Patient  Contact  Record  was  completed  on  all  visits  to  primary 
medical  care  areas2  of  the  four  demonstration  bases  for  one  month  during  1977.s  Patients  were 
asked  to  provide  information  about  themselves  and  about  the  time  it  took  to  obtain  an 
appointment  if  they  had  made  one.  Then,  the  health  provider(s)  furnished  the  medical  details 
of  the  visit.  The  provider  checked  off  the  diagnosis  or  problem  from  a  list  of  the  more  common 
diagnoses  adapted  from  the  ICDA  classification  system  of  the  Royal  College  of  General 
Practitioners,  modified  by  the  University  of  Rochester  for  use  in  training  family  practice 


'These  between-group  comparisons  must  take  statistical  error  into  account.  Appendix  B  discusses  statistical  techniques 
applied  in  these  comparisons. 

^These  areas  included:  adult  primary  care  clinic,  flight  Burgeon's  clinic,  physical  examination  section,  emergency  room, 
and  (if  separate)  internal  medicine  clinic. 

3  A  more  rudimentary  version  of  the  Patient  Contact  Record  had  been  fielded  in  1974  on  visits  to  all  outpatient  clinics, 
at  seven  bases  for  two  weeks,  and  at  two  bases  for  a  six-month  period. 
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physicians,  and  further  modified  by  us.  The  list  was  found  to  account  for  80  to  85  percent  of 
the  diagnoses  in  the  outpatient  clinics  in  our  sample.4  The  categories  for  rating  seriousness 
of  condition  were  taken  from  the  National  Ambulatory  Medical  Care  Survey.  We  devised  our 
own  lists  of  procedures,  tests,  and  medicines. 

Because  of  inherent  limitations  of  the  Patient  Contact  Record  (which  it  shares  with  any 
other  type  of  encounter  form,  and  which  are  described  in  greater  detail  below),  we  confined 
ourselves  to  evaluating  quality  of  care  as  manifested  in  the  technical  process  of  care  (what  is 
done  for  the  patient).  Within  these  bounds,  we  developed  more  than  60  quality  of  care  criteria. 
Some  apply  to  all  persons  with  a  certain  diagnosis  or  problem,  regardless  of  how  many  others 
were  checked,  while  others  apply  only  if  the  one  diagnosis  or  problem  was  checked;  some  apply 
to  every  visit  for  the  diagnosis  or  problem,  while  others  apply  on  the  first  visit  only. 

It  is  worth  noting  in  detail  the  strengths  and  limitations  of  this  encounter  form. 

Its  greatest  advantages  are  that  it  is  completed  on  the  spot,  and  that  the  crucial  informa¬ 
tion  is  furnished  by  the  actual  provider  of  care.  We  observed  conscientious  efforts  by  practi¬ 
tioners  to  complete  the  forms. 

The  form’s  limitations  fall  into  two  categories:  limits  on  the  breadth  of  information,  and 
limits  on  its  reliability.  The  reason  for  visit  was  not  available,5  nor  did  we  have  data  concerning 
the  results  of  laboratory  tests  or  physical  examination.  In  addition,  a  patient  may  have  a 
combination  of  illnesses,  or  an  illness  of  particular  severity  (so  that  a  procedure  justified  for 
one  patient  would  be  inappropriate  for  another).  Various  factors  of  cultural,  psychological, 
social,  economic,  or  geographic  importance  may  suggest  the  wisdom  of  treating  a  disease 
differently  from  the  usual  case,  or  the  provider  may  be  persuaded  to  deviate  from  intended 
care  because  of  the  patient’s  strong  wish  to  omit  or  add  some  procedure.  And,  obviously,  the 
provider  is  limited  to  the  diagnostic  and  procedural  categories  provided  for  checkoff  on  the 
encounter  form  (these  categories  may  not  accurately  describe  some  disease  conditions  or 
services  rendered). 

In  regard  to  reliability,  we  must  rely  on  the  encounter  form  to  convey  a  picture  of  what 
actually  occurred  at  a  particular  visit,  and  we  assume  that  this  picture  represents  a  reasonable 
estimate  of  the  typical  performance  of  these  practitioners.  However,  the  practitioner  may 
have  made  the  wrong  diagnosis,  and  thus  may  have  distorted  our  measurement  by  receiving 
credit  for  "correct”  treatment  of  the  wrong  ailment.  Or  the  provider  may  make  a  recording 
error  when  checking  the  diagnosis  or  procedure  box.  Such  errors  cause  concern  only  if  their 
incidence  is  systematically  related  to  type  of  practitioner.  If  they  occur  randomly  among 
practitioners,  and  are  equally  likely  among  the  three  practitioner  types,  the  performance 
comparisons  will  be  unaffected.  Finally,  practitioners’  performance  may  be  influenced  by 
awareness  of  the  ongoing  study  (the  so-called  "Hawthorne  Effect”). 

Most  of  these  limitations,  rather  than  being  specific  to  the  Patient  Contact  Record,  are 
generic  to  evaluation  of  quality  of  care,  regardless  of  the  techniques  employed.  In  our  opinion, 
none  of  the  limitations,  singly  or  in  combination,  outweighs  the  advantages  of  this  instrument. 
However,  these  limitations  do  underscore  the  need  to  select  simple,  condition-specific  criteria 
or  other  indicators  of  quality,  and  to  interpret  results  cautiously. 


SELECTION  OF  CONDITION-SPECIFIC  CRITERIA 

The  criteria  we  selected  are  simple  in  design,  so  they  can  be  easily  applied  to  the  encounter 
data.  However,  there  are  more  important  reasons  for  using  "simple”  instead  of  "sophisticated” 


Appendix  A  alto  provides  a  frequency  table  of  conditions  marked  on  the  Patient  Contact  Record  by  each  type  of 
practitioner. 

5  An  attempt  to  collect  reason  for  visit  was  made,  but  categories  were  too  few  and  too  broad  to  be  useful 


criteria.  For  one  thing,  they  will  apply  to  a  larger  number  of  patients.  For  another,  simple 
criteria  are  more  likely  to  gain  acceptance  among  providers  and  therefore  carry  much  greater 
weight  if  they  reveal  significant  similarities  or  differences  in  the  quality  of  medical  care 
provided. 

The  authors  accept  responsibility  for  the  medical  content  of  all  the  criteria  selected,  but 
refer  the  reader  to  eleven  sources  of  ambulatory  quality-ofcare  criteria  that  they  found  useful 
in  formulating  the  condition-specific  criteria  that  were  used.  (See  Bibliography,  "References 
for  Quality  of  Care  Criteria.”) 


OTHER  MEASUREMENTS  OF  QUALITY 

Condition-specific  criteria  lie  at  the  heart  of  this  presentation,  but  we  examined  the  quality 
of  care  with  two  other  measures.  The  other  measures  are  not  criteria,  strictly  speaking,  since 
we  did  not  establish  in  advance  normative  statements  of  what  the  care  should  look  like. 

Ordering  rates  for  selected  tests  and  procedures,  which  we  call  utilization,  can  also  be 
construed  as  a  measure  of  quality,  because  unsuitable  ordering  can  not  only  be  wasteful  of 
time,  materials,  and  money  but  also  may  lower  the  quality  of  care.  Utilization  was  our  first 
additional  measure  of  quality. 

Utilization  analysis  examines  the  question  of  appropriate  utilization  of  tests  and  proce¬ 
dures  by  comparing  utilization  rates  for  physicians,  PAs,  and  PCNPs  across  a  number  of  tests 
and  procedures,  including  the  "procedure”  of  ordering  return  visits.  Comparison  of  ordering 
rates  for  each  type  of  practitioner  provides  an  objective  approach  to  this  question.6  The  results 
have  implications  for  both  cost  and  quality.  If  there  are  observed  differences  in  ordering  rates 
among  groups  of  providers,  the  group  ordering  at  a  lower  rate  may  be  "underutilizing”;  or, 
the  group  ordering  at  a  higher  rate  may  be  burdening  the  system  by  "overordering”. 

It  is  also  possible,  however,  that  a  high  rate  of  ordering  is  a  positive  sign,  in  that  it  may 
really  reflect  a  higher  level  of  quality  of  care.  We  thus  quickly  reach  an  ill-defined  zone  where 
quality  and  utilization  are  virtually  inseparable.  It  is  difficult  to  know  whether  differences  in 
ordering  rates  represent  higher  quality  or  lower  quality,  neglect  or  parsimonious  prudence, 
waste  or  merely  reasonable  caution. 

For  both  utilization  and  quality,  utilization  analysis  can  signal  potential  problems,  identi¬ 
fying  areas  where  chart  review  might  permit  more  certain  conclusions  to  be  drawn  concerning 
the  presence  of  high  or  low  quality,  or  too  high  or  too  low  utilization.  Appendix  D  provides 
a  review  of  selected  papers  that  attempt  to  deal  with  the  middle  zone  where  considerations 
of  utilization  and  quality  overlap. 

After  first  presenting  general  data  on  the  ordering  of  tests  and  procedures,  Sec.  IV  below, 
on  utilization  analysis,  probes  more  deeply  into  the  variables  that  may  influence  the  rate  of 
ordering  tests.  One  variable  is  the  diagnostic  mix:  One  type  of  provider  may  overorder  a  given 
test  for  one  diagnostic  group  but,  because  this  type  of  provider  sees  a  smaller  number  of 
patients  in  the  group,  this  type  of  provider  may  end  up  with  the  same  gross  ordering  rate  as 
another  type  of  provider  that  orders  appropriately  for  a  larger  number  of  patients  in  this 
group.  Another  variable  is  the  ratio  of  initial  visits  to  return  visits  for  a  particular  problem. 
If  that  ratio  of  visits  differs  for  physicians  as  opposed  to  PAs  or  PCNPs,  the  rate  of  ordering 
tests  might  well  vary,  since  procedures  are  not  ordered  equally  often  on  initial  and  return 
visits. 

For  the  second  additional  measure  of  quality,  we  analyzed  the  amount  and  character  of 

The  statistical  method  used  to  compare  ordering  rates  was  the  same  method  used  for  the  condition-specific  criteria  and 
described  in  App.  B. 


physician  supervision  of  cases  handled  by  PEs.  Rates  of  consultation  with  and  referral  to 
physicians  were  computed  for  various  categories  of  visits.  This  analysis  does  not  have  physi¬ 
cian  performance  available  as  a  comparative  yardstick;  however,  we  have  drawn  comparisons 
with  findings  from  another  study  of  PE  performance. 


SAMPLE  CONSIDERATIONS 

The  results  presented  in  this  report  rely  on  analysis  of  a  sample  of  visits  collected  at  the 
four  demonstration  base  hospitals.  We  believe  that  the  sample  is  reasonably  random  and  thus 
gives  a  reliable  picture  of  the  care  provided.  We  also  believe  that  the  demonstration  bases  are 
representative  of  other  Air  Force  hospitals  of  similar  size,  and  that  the  results  obtained  in  this 
study  would  apply  to  these  other  hospitals. 

Clinic  personnel  collected  data  using  the  PCR  on  all  visits  for  a  four-week  period.  Table 
1  shows  the  sample’s  time  period  and  the  number  of  records  collected  at  each  base  in  the 
primary  medicine  clinics  (including  family  practice,  general  therapy,  flight  surgeon,  physical 
exam,  emergency  room,  walk-in,  sick  call,  and  general  internal  medicine  clinics).  If  direct 
activity  in  the  sample  period  is  typical,  it  should  be  representative  of  the  year  as  a  whole;  we 
have  no  evidence  that  it  was  not.7 

We  believe  that  the  demonstration  bases  and  personnel  are  reasonably  representative 
of  the  Air  Force  Medical  Service.  The  bases  were  chosen  from  a  variety  of  commands  and  were 
typical  of  middle-sized  Air  Force  hospitals.  Bases  were  also  selected  that  were  likely  to  under¬ 
go  as  few  transfers  of  personnel  as  possible,  so  as  to  establish  a  stable  manning  ratio  for  the 
demonstration.  This  meant  that  the  demonstration  bases  already  had  fewer  physicians  and 
more  PEs  than  the  typical  base  hospital.  New  physicians  were  added  to  the  hospitals  in  the 
course  of  normal  rotation.  New  PEs  were  usually  assigned  from  the  newly  graduated  class  of 
PAs,  with  no  attempt  to  choose  only  the  better  graduates. 


Table  1 

Basic  Data  on  Primary  Medicine  Clinics  Recorded  During 
Sample  Period,  1977 

Item  Chanute  Dyess  Fairchild  Nellis 

Total  number  of  MDs  12  8  15  13 

Panel  PAs  7  S  5  6 

Panel  PCNPs  1222 

Sample  collection  dates  3-30/4-26  3-17/4-13  4-27/5-27®  4-13/5-11 

No.  of  patient  visits  5010  4304  3780  4959 

aExcludes  Hay  3,  4,  and  5,  1977,  which  were  dedicated  to  health 
screening  examinations. 


Because  of  the  relatively  small  number  of  PCNPs,  however  (see  Table  1),  conclusions  about 
their  performance  cannot  be  confidently  applied  to  PCNPs  in  the  rest  of  the  Air  Force.  The 
larger  numbers  of  PAs  and  physicians  give  estimates  of  performance  that  could  more  reliably 
be  expected  to  apply  to  the  Air  Force  as  a  whole. 

7 During  the  sample  period  at  Fairchild,  three  days  of  the  clinic  time  were  dedicated  for  health  screening  exams.  These 
days  were  removed  from  the  sample  and  the  four-week  period  was  extended  an  additional  three  days. 


III.  CONDITION-SPECIFIC  CRITERIA 


BACKGROUND 

We  first  turn  to  an  evaluation  of  quality  of  care  through  the  use  of  condition-specific 
criteria.  The  policy  issue  is  whether  the  Air  Force  can  deliver  the  same  quality  of  medical  care 
when  physician’s  assistants  (PAs)  and  primary  care  nurse  practitioners  (PCNPs),  working 
under  the  general  supervision  of  a  physician,  provide  a  substantial  portion  of  the  outpatient 
care  formerly  provided  by  physicians. 

This  section  compares  the  physician’s  extenders’  (PEs’)  quality  of  care  with  that  of  physi¬ 
cians,  for  routine  outpatient  conditions  that  the  PEs  are  trained  to  treat.1 

We  use  the  descriptive  data  collected  on  Patient  Contact  Records  for  a  one-month 
period  at  the  four  demonstration  bases.  The  data  come  from  the  group  of  selected 
clinics  at  each  base  that  provide  primary  care  adult  medicine.2 


SELECTION  OF  CRITERIA 

Section  II  has  discussed  strengths  and  weaknesses  of  our  4  a  collection  instrument,  the 
Patient  Contact  Record.  By  concentrating  on  routine  cases  of  low  or  moderate  complexity 
(which  account  for  most  cases  seen  in  Air  Force  outpatient  clinics),  we  believe  we  avoided  the 
difficulties  and  disagreements  that  sophisticated  criteria  probably  would  have  engendered. 
Measures  of  technical  process  make  sense  in  the  outpatient  setting,  where  many  conditions 
being  treated  are  common  and  conventionally  managed,  and  where  there  is  a  relatively  clear 
connection  between  process  of  care  and  outcome  (cure  or  symptom  relief).  Finally,  we  used 
team  physicians’  compliance  with  the  same  criteria  as  a  benchmark  against  which  to  judge 
the  quality  of  care  rendered  by  team  extenders.  The  physicians  were,  of  course,  working  in 
the  same  primary  medicine  settings. 


CLASSES  OF  CRITERIA 

We  divided  our  criteria  into  five  classes:  desirable  diagnostic  actions,  desirable  therapeutic 
actions,  undesirable  diagnostic  actions,  undesirable  therapeutic  actions,  and  desirable  disposi¬ 
tion  actions.  Table  2  provides  an  example  from  each  class,  and  shows  the  format  in  which  we 
present  the  results  of  all  our  condition-specific  criteria.  The  word  "only”  in  the  second,  third, 
and  fourth  column  heads  signifies  that  we  limited  the  application  of  our  criterion  to  those 
visits  where  it  was  clear  that  only  one  MD,  PA,  or  PCNP  provided  the  care.  (We  thus  excluded, 
for  example,  the  rare  visits  when  physicians  and  PAs  together  provided  care.) 

For  each  type  of  provider,  Table  2  documents  a  percentage  of  compliance  with  the  criterion 
and  the  number  of  visits  to  which  the  criterion  was  applied.  (Note:  If  the  criterion  is  specifically 
limited  to  first  visits,  the  number  of  visits  equals  the  number  of  patients  involved;  however, 
if  the  criterion  is  applied  to  all  visits  within  a  given  diagnostic  category,  the  number  of  patients 
is  likely  to  be  smaller  than  the  number  of  visits.  The  n  that  we  record  is  the  number  of  visits.) 


1  See  App.  A  for  the  complete  list  of  conditions  and  the  frequencies  with  which  they  were  marked  by 
each  type  of  practitioner. 

^Some  of  our  analysis  pools  the  data  collected  at  the  four  bases,  a  reasonable  procedure  since  there  was  no  major  variation 
from  base  to  baBe.  Exceptions  to  the  finding  of  lack  of  significant  variation  are  discussed  in  Apps.  B  and  C. 


The  last  column  of  Table  2  indicates  whether  between-group  differences  were  statistically 
significant  at  the  P  <  0.05  level,  using  the  test  for  difference  in  proportions.3  In  other  words, 
the  chances  of  getting  the  measured  difference  or  a  larger  difference  between  practitioner 
types  would  be  5  percent,  if  the  practitioners  were  actually  complying  at  equal  levels.  When 
there  is  a  statistically  significant  difference,  it  is  recorded  by  showing  which  group  or  groups 
of  providers  have  a  percentage  compliance  with  the  criterion  significantly  greater  than  the 
other(s).  In  the  case  of  desirable  actions,  a  higher  percentage  is  consistent  with  better  care. 
However,  as  will  be  emphasized  in  greater  detail  below,  in  the  case  of  undesirable  actions,  a 
higher  percentage  represents  worse  care. 


REPRESENTATIVE  CRITERIA 

The  first  criterion  says  that  if  the  patient  made  a  first  visit  for  a  problem  diagnosed  by 
the  provider  of  care  as  urinary  tract  infection,  the  provider  then  ordered  either  a  urinalysis 
or  a  urine  culture. 

Under  desirable  therapeutic  action,  the  sample  criterion  reads:  If  the  patient  made  a  first 
visit  which  was  diagnosed  by  the  provider  as  infectious  otitis  media,  then  the  provider  pre¬ 
scribed  an  appropriate  class  of  antibiotic  (i.e.,  penicillin,  erythromycin,  or  sulfa). 

Through  undesirable  diagnostic  actions,  we  can  approach  the  question  of  "overordering” 
by  various  groups  of  providers.  Note  that  because  the  action  is  undesirable,  a  lower  percentage 
suggests  a  better  ordering  pattern.  For  example,  are  practitioners  ordering  electrocardio¬ 
grams  for  a  group  of  diagnoses  where  electrocardiograms  are  unlikely  to  have  been  needed? 

Once  again,  with  undesirable  therapeutic  actions,  a  lower  percentage  implies  better  care, 
because  the  action  is  undesirable.  The  representative  criterion  reads:  If  the  provider  selected 
the  diagnosis  of  viral  syndrome  with  gastroenteritis,  then  an  antibiotic  was  prescribed.  In  this 
instance,  because  it  is  considered  poor  medicine  to  prescribe  an  antibiotic,  a  lower  percentage 
compliance  represents  better  care. 

A  group  of  desirable  disposition  actions  constitutes  our  final  criterion.  The  representative 
criterion  reads:  If  a  tranquilizer  or  antidepressant  was  prescribed,  then  a  definite  follow-up 
visit  was  planned. 


BREADTH  OF  CRITERIA 

It  is  of  interest  to  know  the  "amount  of  care”  upon  which  our  technical-process,  quality-of- 
care  criteria  impinge.  Although  each  criterion  views  only  one  aspect  of  the  care  of  a  particular 
problem,  and  therefore  does  not  thoroughly  evaluate  care  for  each  condition,  the  full  set  of 
criteria  taken  as  a  whole  does  evaluate  a  substantial  portion  of  care  provided  by  each  group 
of  practitioners. 

We  have  computed  the  percentages  of  all  visits  to  team  members  for  conditions  specifically 
evaluated  by  our  criteria.*  The  results  are  presented  in  Table  3.  By  "specifically  evaluated,” 
we  mean  that  the  condition  was  mentioned  by  name  on  our  list  of  criteria.  Thus,  for  two 
reasons,  the  reported  percentage  underestimates  the  total  amount  of  care  evaluated:  first,  the 
denominator  includes  routine  examinations,  whose  content  is  fixed  and  to  which  criteria  could 
not  be  applied;  second,  we  did  not  add  to  the  numerator  the  visits  for  various  groups  of 


■*Thi8  and  other  testa  for  statistical  significance  are  discussed  in  App.  B. 

*The  complete  list  of  condition-specific  criteria  upon  which  we  base  our  conclusions  is  found  in  Tables  C.l  to  C.5.  App. 
C.  We  report  on  62  criteria  in  all:  17  desirable  diagnostic  actions,  14  desirable  therapeutic  actions,  8  undesirable  diagnostic 
actions,  16  undesirable  therapeutic  actions,  and  7  desirable  disposition  actions. 


conditions  (for  example,  the  "follow-up  desirable”  group  of  conditions— see  the  list  of  groups 
of  conditions  in  Table  C.6,  App.  C). 


Table  3 

Amount  of  Care  Evaluated  by  1977  Criteria 


Type  of  Provider 

Number  of  Visits 
to  Which  Criteria  Apply 
Divided  by 

Total  Number  of  Visits 

%  Visits 
Evaluated  by 
the  1977  Criteria 

Total  visits  to 

individual  team  members 

5919/14,711 

40.2% 

Visits  to  MD  only 

1691/4429 

38.2% 

Visits  to  PA  only 

3280/8227 

39.9% 

Visits  to  PCNP  only 

948/2055 

46.1% 

Thus,  depending  on  the  mix  of  conditions  seen  by  each  provider,  the  criteria  were  applica¬ 
ble  to  between  38  percent  and  46  percent  of  all  visits  seen  individually  by  providers.  For  the 
group  as  a  whole,  the  1977  criteria  were  applicable  to  approximately  40  percent  of  all  visits.5 


SUMMARY  COMPARISON 

Having  reviewed  the  five  classes  of  quality  of  care  criteria,  we  turn  our  attention  to  the 
summary  comparison,  which  reflects  the  criteria  listed  in  App.  C.  We  first  chose  42  of  the  total 
of  62  criteria  in  constructing  the  summary,  and  gave  each  criterion  equal  weight.  A  choice  was 
made  on  which  criterion  to  count  when  two  or  more  criteria  overlapped,  and  we  attempted 
to  use  "best  medical  judgment”  in  making  the  choice.  We  then  eliminated  any  criterion  that 
had  too  few  cases  to  be  able  to  detect  a  true  difference  of  0.25  in  compliance  rates  at  least  75 
percent  of  the  time.® 

The  basic  question  was:  How  are  PAs  and  PCNPs  measuring  up,  insofar  as  we  can  deter¬ 
mine  with  our  criteria?  Is  their  level  of  performance  either  equal  to  or  better  than  the 
standard,  i.e.,  compliance  by  team  physicians  with  the  same  criteria?  The  results  were  strik¬ 
ing.  Very  few  significant  differences  between  the  provider  groups  emerged  from  the  compari¬ 
son.  Directing  our  attention  to  the  bottom  line  of  Table  4,  entitled  'Total  Actions”,  we  see  that, 
for  5  of  the  28  criteria,  PAs’  performance  was  (statistically)  significantly  superior  to  that  of 
physicians.  For  20  of  the  28,  there  was  no  difference  between  the  performance  of  the  PAs  and 
the  physicians.  Therefore,  for  25  out  of  28  quality  criteria,  PAs’  performance,  on  a  statistical 
basis,  equaled  or  exceeded  the  performance  standard  we  set.  PAs  performed  worse  on  3 
criteria  out  of  28. 

Performance  of  the  PCNPs  was  (statistically)  significantly  superior  to  that  of  physicians 


This  percentage  compares  with  the  20  to  26  percent  of  visits  to  which  we  were  able  to  apply  a  more  limited  set  of  criteria 
in  ap  earlier  study  of  PA  quality  discussed  below. 

This  standard  reflects  the  power  of  the  statistical  test.  The  calculation  of  each  test’s  power  is  discussed  in  App.  B. 


Table  4 


Summary  Comparison:  Quality  of  Care  in  Primary 
Medical  Settings,  Demonstration  Bases,  1977 


Criterion  Class 

PA 

Better 

PA 

Equal 
to  MD 

MD 

Better 

PCNP 

Better 

PCNP 
Equal 
to  MD 

MD 

Better 

Desirable  diagnostic  action 

i 

5 

0 

1 

2 

0 

Desirable  therapeutic  action 

2 

3 

1 

1 

0 

1 

Undesirable  diagnostic  action 

0 

3 

1 

1 

2 

1 

Undesirable  therapeutic  action 

1 

7 

0 

0 

7 

1 

Desirable  disposition  action 

1 

2 

i 

0 

0 

2 

Total  actions 

5/28 

20/28 

3/28 

3/19 

11/19 

5/19 

MOTE:  The  significance  tests  for  this  table  are  based  on  the  critical 
value  for  two  contrasts.  Tests  using  the  single-contrast  critical  value 
for  MD-PA  comparisons  give  similar  results:  Based  on  total  actions,  PAs 
perform  better  on  6  out  of  28  criteria  and  equal  MD  performance  on  19  out 
of  28;  MD  performance  excels  PA  performance  on  3  out  of  28. 


for  three  of  the  19  measures,7  and  was  (statistically)  the  same  for  11  of  the  19  criteria. 
Therefore,  for  a  total  of  14  out  of  19  quality  criteria,  PCNPs’  performance  equaled  or  exceeded 
the  performance  standard.  PCNPs  performed  worse  on  5  criteria  out  of  19. 

Another  measure  of  differences  in  overall  performance  can  be  calculated  by  using  the  sign 
test.  Summarized  in  App.  B,  the  results  of  this  test  show  that  PCNPs  do  not  differ  significantly 
from  physicians;  in  21  of  42  non-overlapping  criteria,  PCNPs  outperform  physicians.  However, 
PAs’  compliance  with  criteria  exceeds  physicians’  compliance  on  28  of  the  42  criteria,  a  result 
unlikely  to  occur  (p  <  .05)  if  PAs  and  physicians  were  performing  at  the  same  level. 

Table  5  lists  those  criteria,  from  among  those  used  in  the  summary  comparison,  on  which 
PEs  and  physicians  excelled  each  other. 

Inspection  of  the  small  number  of  significant  differences  in  performance  reveals  a  feature 
that  is  medically  reassuring:  There  is  a  lack  of  systematic  deficiency  on  the  part  of  any  group 
of  providers.  That  is,  the  few  differences  that  there  are,  are  scattered  among  the  five  classes 
of  criteria.  Upon  comparison  of  physicians  and  PAs,  the  3  criteria  according  to  which  physi¬ 
cians  performed  better  are  scattered  among  three  classes;  so  are  the  5  criteria  according  to 
which  PAs  performed  better.  For  physicians  and  PCNPs,  the  5  criteria  according  to  which 
physicians  performed  better  are  scattered  among  four  classes;  the  3  criteria  according  to  which 
PCNPs  performed  better  are  scattered  among  three  classes.  Because  we  believe  that  each 
criterion  has  medical  merit  (although  the  criteria  are  not  all  of  equal  importance  or  validity), 
it  is  impossible  to  say  that  any  criterion  showing  a  difference  is  "unimportant”;  yet,  the 
predominance  of  criteria  showing  equal  or  superior  performance  by  PEs  is  the  key  finding, 
and  the  typical  criterion  showing  similar  or  superior  performance  is  certainly  equal  in  "impor¬ 
tance”  to  the  typical  criterion  showing  inferior  performance. 

Additional  analysis  leads  to  further  insight.  For  example,  significant  differences  in  the 
performance  of  PAs  and  PCNPs  occurred  for  only  three  of  the  criteria  (two  of  which  were  used 


7The  number  of  criteria  for  PCNPa  is  less  than  that  for  PA*  because  more  criteria  were  discarded  because  of  insufficient 
power  in  the  tests— the  result  of  fewer  PCNP  visits. 


Table  5 


Criteria  on  Which  PE  and  Physician  Performance 
Differed  Significantly:  Primary  Medical 
Settings,  Demonstration  Bases,  1977 


MDs  performed  significantly  better  tnan  PAs: 

Urinary  test  infection  (only  diagnosis,  first  visit)  antibiotic 
prescribed. 

Coryza  (only  diagnosis,  first  visit)  throat  culture  ordered. 

(Lower  percentage  is  desirable.) 

Condition  that  should  usually  be  followed  up  -*•  definite  return 
appointment  planned. 

MDs  performed  significantly  better  man  PC  DPs : 

Urinary  tract  infection  (only  diagnosis,  first  visit)  -*■  antibiotic 
prescribed. 

Backache  alone  or  with  sciatica  (only  diagnosis,  first  visit)  -*■  x-ray 
ordered.  (Lower  percentage  is  desirable.) 

Viral  syndrome  with  gastroenteritis  antibiotic  prescribed.  (Lower 
percentage  is  desirable.) 

Condition  judged  "very  serious/serious"  by  provider  •+  definite 
follow-up  planned. 

Condition  that  should  usually  be  followed  up  -►  definite  return 
appointment  planned. 

PAs  performed  significantly  Letter  than  MDs: 

Pharyngitis  (only  diagnosis,  first  visit)  throat  culture  ordered 
or  penicillin  prescribed. 

Acute  sinusitis  (first  visit)  ■*  any  medication  prescribed. 

Coryza  or  febrile  cold  -*■  antibiotic  prescribed.  (Lower  percentage 
is  desirable.) 

Infectious  otitis  media  (first  visit)  -*  decongestant  prescribed. 

Infectious  otitis  media  (only  diagnosis,  first  visit)  *♦  definite 
follow-up  planned. 


PCNPs  performed  significantly  better  tnan  MDs: 

Pharyngitis  (only  diagnosis,  first  visit)  ■+  throat  culture  ordered 
or  penicillin  prescribed. 

Infectious  otitis  media  (first  visit)  ■*  decongestant  prescribed. 


No.  of  EKGs  onyoung  adults  without  "EKG  condition” 


No.  of  visits  for  other  than  "EKG  condition 


Lower  fraction 


is  desirable. 
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in  the  summary  comparison): 

•  Otitis  media  (infectious,  first  visit):  antibiotic  prescribed 

•  Backache  (only  diagnosis,  first  visit):  x-ray  ordered 

•  Condition  judged  "very  serious) serious” :  definite  followup  planned 

For  all  three  of  these  criteria,  PAs  performed  better.8  For  59  other  criteria,  however,  there 
was  no  significant  difference  in  performance  between  these  two  groups  of  extenders.  We 
therefore  conclude  that  PAs  and  PCNPs  perform  about  the  same. 


COMPARISON  OF  1977  AND  1974  RESULTS 

In  1974,  using  similar  methods,  we  measured  quality  of  care  rendered  by  PAs.9  Only  a  few 
of  the  criteria  applied  in  1977  had  also  been  applied  in  1974 — a  more  limited  Patient  Contact 
Record  in  1974  constrained  the  choice  of  criteria.  Similar  but  not  exactly  the  same  criteria 
were  formulated  in  1974  for  otitis  media  (a  prescription  needed  on  the  first  visit),  acute 
sinusitis  (a  prescription  needed  on  the  first  visit),  urinary  tract  infection  (a  prescription  needed 
on  the  first  visit),  and  pneumonia  (a  chest  x-ray  needed  on  the  first  visit).  For  two  other 
criteria,  pharyngitis  (throat  culture  needed  on  first  visit),  and  diabetes  (blood  sugar  needed 
on  first  visit),  the  criteria  in  1974  and  1977  were  virtually  identical. 

When  1977  results  are  compared  with  1974  results  (Table  6),  there  is  no  evidence  of  any 
worsening  in  PAs’  performance — despite  PA  classes  having  been  trained  at  different  times,  a 
number  of  PAs’  having  been  out  of  school  for  a  longer  time,  and  a  higher  ratio  of  PAs  to 
supervising  physicians.  This  finding  is  encouraging. 

Data  were  also  available  to  explore  further  the  question  of  differences  in  performance 
among  various  cohorts  of  PAs.  (1974  data  did  not  include  information  on  PCNPs.)  Analysis  of 
individual  performance  on  criteria  can  show  whether  differences  in  individual  performance 
rates  are  related  to  ai'<y  known  characteristics  of  individual  practitioners.  One  characteristic 
of  interest  is  year  of  PAs’  training.  By  comparing  the  performance  of  PAs  who  graduated  from 
the  Air  Force’s  training  program  before  June  1975  with  those  who  graduated  after  June  1975, 
we  can  test  whether  the  performance  levels  in  1977  were  maintained,  both  by  the  cohort  of 
early  PA  graduates  and  by  the  new  cohorts  of  graduates  entering  the  PA  force. 

Table  7  shows  PAs’  performance  on  a  selection  of  criteria  comparing  two  cohorts  of  PAs. 
The  "New  PA"  group  includes  those  who  graduated  from  the  training  program  after  June 
1975;  the  "Old  PA”  group  consists  of  those  trained  before  June  1975.  There  was  little  difference 
in  their  performance.  Recent  PA  graduates  perform  as  well  as  earlier  graduates,  as  measured 
by  compliance  with  these  criteria.  These  thirteen  criteria  are  broadly  representative  of  the 
complete  list  of  criteria.  Criteria  were  chosen  to  show  a  range  of  PAs’  performance — criteria 
where  the  PAs  both  exceed  and  fail  to  meet  the  physicians’  performance  levels.  The  sample 
sizes  for  each  criterion  had  to  be  large  enough  to  produce  adequate  statistical  power,  and 
criteria  were  also  chosen  to  include  selections  from  each  of  the  five  categories  of  criteria. 


8The  critical  value  (d  =  2.24)  used  in  these  PA-PCNP  comparisons  was  the  same  as  used  in  the  physician-PA  and  physician- 
PCNP  comparisons,  although  the  addition  of  another  contrast  (PA-PCNP)  should  require  a  higher  critical  value.  Since  we 
are  primarily  interested  in  the  physician-versus^xtender  comparison,  the  critical  value  was  choaen  baaed  on  these  two 
contrasts.  The  same  value  was  used  here  to  maintain  comparability. 

9N-1184-AF,  'The  Quality  of  Air  Force  Outpatient  Care:  How  Well  Do  Physician  Assistants  Perform?” 
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Table  6 


Comparison  of  PA  Performance  in  1974  and  1977 


Criterion 

1974 

Rate 

1977 

Rate 

Statistically 

Significant? 

1. 

Diabetes  Mellitus  (first  visit); 
blood  sugar  ordered  (all  visits 
included  in  1977) 

29% 
n=  7 

51% 
ns  102 

no 

5. 

Pharyngitis  (only,  first  visit); 
throat  culture  ordered 

70% 

n=155 

84% 

n=223 

1977  >  1974 

7. 

Urinary  tract  infection  (only, 
first  visit);  urine  culture  or 
urinalysis  ordered 

79% 

n=77 

75% 

n=69 

no 

12. 

Pneumonia  (first  visit);  chest 
x-ray  ordered  (sputum  culture 
included  in  1977) 

67% 

n=l2 

80% 

n=20 

no 

19. 

Acute  sinusitis  (first  visit); 
any  medication  prescribed 

93% 

n-55 

98% 

n=60 

no 

23. 

Otitis  media  (first  visit);  any 
medication  prescribed  (limited 
to  antibiotics  in  1977) 

91% 

n=180 

94% 

n-100 

no 

27. 

Urinary  tract  infection  (only, 
first  visit);  any  medication 
prescribed 

77% 

n=77 

64% 

n=69 

no 

CONCLUSION 

Despite  the  limitations  of  condition-specific  measurements  of  quality,  there  is  strong  evi¬ 
dence  that,  for  the  kinds  of  conditions  that  they  are  trained  to  treat,  both  PAs  and  PCNPs 
perform  at  an  acceptable  level  of  quality,  when  physicians’  compliance  with  the  same  quality 
criteria  is  employed  as  a  standard  of  reference. 10  We  find  that  the  extenders  deliver  acceptable 
quality  of  care  in  this  setting.  Insofar  as  we  can  measure  with  our  condition-specific  criteria, 
PEs  are  safe  for  routine,  outpatient  conditions,  in  that  they  deliver  a  technical  process  of  care 
equal  to  that  of  physicians. 


10We  wish  to  add  that  these  objective  results  are  consistent  with  the  high  subjective  ratings  given  to  these  extenders  by 
their  supervisors. 


Table  7 

Comparison  of  PA  Cohorts  Graduating  before  and  after  June  1975 


n 


Criterion 

"Old  PA" 

"New  PA" 

Statistically 

Significant? 

3. 

Hypertension  (only) ; 
blood  pressure  taken 

56% 

n*=l68 

65% 

n=182 

Yes 

6. 

Pharyngitis  (only,  first  visit); 
throat  culture  or  penicillin 
prescribed 

92% 

^64 

91% 

n=160 

No 

7. 

Urinary  tract  infection  (only, 
first  visit);  urine  culture  or 
urinanalysis  ordered 

70% 

n«27 

79% 

n=42 

a 

tio 

18. 

Acne  treated  with  antibiotic; 
antibiotic  is  tetracycline 

72% 

n=36 

80% 

n=81 

Wo 

24. 

Otitis  media  (infectious;  first 
visit);  penicillin,  erythromycin, 
or  sulfa 

90% 

n=40 

93% 

n=6l 

Ho 

2b. 

Otitis  media  (noninfectious ,  first 
visit) ;  decongestant  prescribed 

83% 

n=24 

92% 

n-108 

Ho 

28. 

Urinary  tract  infection  (only, 
first  visit) ;  antibiotic  prescribed 

37% 

n=27 

45% 

n=42 

ttoa 

36. 

Backache  alone  or  with  sciatica 
(only,  first  visit);  x-ray  ordered 

8% 

n=25 

17% 

n=47 

Noa 

39. 

Coryza  (only,  first  visit); 
throat  culture  ordered 

49% 

n=85 

51% 

n=217 

Ho 

41. 

Coryza  or  febrile  cold; 
antibiotic  prescribed 

9% 

n*241 

10% 

n=465 

Ho 

42. 

Viral  syndrome  without  gastro¬ 
enteritis;  antibiotic  prescribed 

14% 

n=105 

15% 

n=94 

Ho 

51. 

Ho.  of  times  oral  or  injected 
steroid  was  prescribed,  divided 
by  no.  of  visits  for  "steroid- 
contraindicated  conditions" 

0.2% 

n=433 

0.2% 
n=52  7 

Ho 

56. 

Condition  judged  very  serious/ 
serious  by  provider;  definite 
follow-up  planned 

87% 

n=55 

78% 

n=206 

No 

aThe  power  of  these  tests  failed  to  meet  the  criteria  of  at  least  75- 
percent  chance  of  detecting  a  difference  of  0.25  in  the  true  compliance 


IV.  UTILIZATION  ANALYSIS 


INTRODUCTION 

Appropriate  ordering  of  tests  and  procedures  is  another  important  facet  of  good  quality 
medical  care.  A  portion  of  the  previous  section  examined  undesirable  diagnostic  actions  and 
undesirable  therapeutic  actions  by  selecting  specific  tests,  procedures,  or  prescriptions  (e.g., 
electrocardiogram,  physical  therapy,  steroids)  and  analyzing  whether  or  not  they  were  em¬ 
ployed  in  situations  where  they  were  likely  to  have  been  inappropriate. 

This  section  adopts  a  different  approach:  "utilization  analysis.”  As  explained  in  Sec.  II, 
results  of  utilization  analysis  can  have  implications  for  both  cost  and  quality.  However,  we  pay 
particular  attention  to  the  question  of  higher  ordering  rates  as  a  sign  of  possible  overutiliza¬ 
tion,  because  of  the  often  expressed  fear  that  physician’s  extenders  (PEs)  might  overburden 
a  system  by  overordering.  Although  we  raise  the  issue  of  quality  from  time  to  time  as  we 
review  the  data,  we  give  relatively  more  weight  to  considerations  of  cost. 

This  section  presents  data  on  the  ordering  of  tests  and  procedures — first  in  general,  and 
then  for  specific  diagnoses  and  conditions.  It  next  examines  the  pattern  of  "disposition,”  the 
decision  made  at  the  end  of  every  visit  as  to  what  kind  of  follow-up  will  be  planned.  Disposition 
is  treated  in  general  only,  because  the  observed  pattern  is  clear  and  explicable.  Finally,  we 
reach  the  conclusion  that  no  serious  weaknesses  are  discernible  in  the  utilization  patterns  of 
PEs,  and  we  briefly  discuss  the  implications  of  this  finding. 


METHODS 

The  data  collection  methods  were  described  in  Sec.  II,  but  to  repeat:  Data  collected  on 
Patient  Contact  Records  during  the  1977  survey  were  used.  Data  are  presented  for  patients 
seen  only  by  a  physician,  or  by  a  physician’s  assistant  (PA),  or  by  a  primary  care  nurse 
practitioner  (PCNP),  We  have  excluded  tests  ordered  during  the  infrequent  visits  when  a 
patient  saw  either  none  or  more  than  one  of  these  three  types  of  providers.  For  all  tables, 
statistical  significance  is  tested  at  a  level  of  p  <  0.05.  That  is,  we  say  that  the  difference  between 
two  practitioners  is  significant  when  the  chance  of  measuring  such  a  difference  occurring  in 
a  sample  of  visits  is  less  than  5  percent  if  the  two  practitioners  were  actually  ordering  at  the 
same  rate.  We  examine  the  appropriateness  of  utilization  by  computing  the  number  of  times 
a  specific  test  or  procedure  was  ordered  during  100  visits  for  all  conditions,  or  for  a  selected 
group  of  diagnoses.1  Although  it  might  be  unwise  to  make  value  judgments  concerning  the 
suitable  number  of  times  that  ordering  should  occur,  we  would  be  troubled  if  physicians,  PAs, 
and  PCNPs  were  ordering  at  widely  varying  rates.  These  rates  are  also  helpful  to  the 
administrator  who  wants  to  predict  the  number  of  tests  that  he  can  expect  the  staff  to  order. 


ORDERING  FOR  ALL  CONDITIONS 

We  computed  the  rate  of  ordering  for  the  following  tests  or  procedures:  any  type  of  x-ray, 
physical  therapy,  electrocardiogram,  urinalysis,  electrolytes,  and  blood  counts.  (The  entire  set 


'Table  C.6  in  App.  C  specifies  the  diagnoses  that  comprise  each  group. 
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of  visits  was  considered,  not  merely  the  40  percent  of  visits  covered  by  the  criteria  in  the 
previous  section.)  Table  8  shows  the  gross  rate  of  ordering  these  tests  or  procedures  per  100 
visits  for  patients  seen  only  by  a  physician,  or  only  by  a  PA,  or  only  by  a  PCNP.  Upon  review 
of  the  rates,  it  is  apparent  that  there  are  no  large  absolute  differences  in  rates  of  ordering, 
for  all  problems  taken  as  a  group.  For  example,  with  the  blood  count  category,  we  find  that 
PAs  are  ordering  approximately  one-half  count  more  per  100  visits  more  than  are  physicians 
or  PCNPs.  This  difference  would  not  seem  to  indicate  misordering:  It  is  not  statistically 
significant  and  is  unlikely  to  be  clinically  or  economically  significant,  either.  With  urinalysis, 
the  PAs  are  ordering  approximately  one  and  a  half  more  urinalyses  per  100  visits  than  are 
physicians.  Even  though  this  difference  is  significant,  it  is  doubtful  that  any  clinical  or  econom¬ 
ic  significance  can  be  attached  to  it. 


Table  8 

Rates  of  Ordering  Tests  per  100  Visits,  for  all  Conditions 


Test 

MD 

n*4429 

PA 

n-8227 

PCUP 

n=2055 

Statistically 

Significant? 

Any  x-ray 

9.60 

11.6 

n.i 

yes 

Physical  therapy 

0.23 

1.61 

1.46 

yes 

Electrocardiogram 

3.66 

1.65 

1.22 

yes 

Urinalysis 

6.05 

7.72 

7.30 

yes 

Electrolytes 

2.91 

2.80 

3.75 

no 

Complete  blood  count 

white  blood  count, 

hematocrit 

7.42 

8.00 

7.49 

no 

Two  rather  marked  differences  are  apparent,  and  they  occur  in  opposite  directions:  PAs 
are  ordering  approximately  two  more  x-rays  per  100  patient  visits  than  are  physicians,  while 
physicians  are  ordering  approximately  two  more  electrocardiograms  per  100  visits  than  are 
PAs.  In  both  cases,  the  ordering  rate  for  PCNPs  is  quite  close  to  that  of  PAs.  Both  of  these 
findings  are  statistically  significant.  One  could  conjecture  about  possible  clinical  significance 
(and  we  will  do  so  below),  but  the  economic  importance  of  these  differences  is  likely  to  be  minor 
because  of  the  low  marginal  cost  of  additional  x-rays  and  electrocardiograms  in  Air  Force 
facilities. 

Taking  the  categories  together,  we  note  that  for  three  of  these  tests,  PAs  order  significant¬ 
ly  more  tests;  for  one  of  them,  PNCPs  order  significantly  more;  and  for  another,  physicians 
order  significantly  more.  Throughout  this  discussion,  tests  of  statistical  significance  are  fash¬ 
ioned  in  order  to  compare  the  two  types  of  PEs  with  physicians,  and  not  with  each  other. 


ORDERING  FOR  SPECIFIC  CONDITIONS 

The  gross  calculations  above  do  not  probe  the  variables  that  may  influence  the  ordering 
of  tests.  They  do  not  really  tell  us  if  PAs  or  PCNPs  are  overutilizing  tests  and  procedures  in 
comparison  with  physicians.  Only  when  variables  such  as  diagnostic  mix  and  the  proportion 
of  initial  visits  to  return  visits  are  taken  into  account  can  we  perform  a  more  exacting  analysis 
of  different  rates  of  ordering  tests.  Even  then,  the  reader  will  observe  that  we  are  left  with  the 
question  of  the  reason  for  any  observed  differences  in  ordering  rates:  We  reach  that  ill-defined 
zone  where  quality  and  utilization  are  inseparable.  Nevertheless,  we  acquire  useful  informa- 
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tion  on  utilization  of  services,  and  we  signal  potential  problem  areas  relating  to  both  utilization 
and  quality. 

The  tables  that  follow  specify  a  narrower  range  of  diseases,  in  order  to  lessen  the  effect 
of  diagnostic  mix  on  the  results.  Likewise,  they  separate  first  visits  from  return  visits. 


ORDERING  OF  X-RAYS 

Turning  to  disease-specific  rates  of  ordering  tests,  we  start  with  x-rays  (Table  9).  The  top 
section  of  Table  9  controls  for  disease  categories.  "X-ray  conditions”  is  the  general  term  for 
a  group  of  disorders,  all  of  which  might  well  require  x-rays  at  some  time  for  some  proportion 
of  patients  being  seen.  The  five  more  specific  disease  conditions  listed  (i.e.,  headaches,  ischemic 
and  other  heart  diseases,  acute  and  chronic  sinusitis,  backache,  and  arthritis  and  joint  pain) 
are  all  specific  subsets  of  the  general  term  "x-ray  conditions.”  This  table  limits  the  count  of 
visits  to  those  encounters  for  which  the  specific  diagnosis  was  the  sole  one  written  on  the 
Patient  Contact  Record. 

Now  that  we  have  taken  case  mix  into  account,  we  notice  dramatic  changes  in  the  num¬ 
bers.  Before  (Table  8),  both  PAs  and  PCNPs  had  ordered  more  x-rays  than  physicians  per  100 
visits  for  all  conditions.  We  now  note  an  even  larger  difference  (and  a  statistically  significant 
one)  in  ordering  rate  when  we  look  at  the  more  specific  groups  of  diagnoses.  In  every  case  for 
PAs,  and  in  every  case  but  one  for  PCNPs,  these  two  types  of  PEs  are  ordering  more  x-rays 
than  are  physicians  for  each  of  the  diagnostic  subsets;  and  the  PCNPs  are  ordering  significant¬ 
ly  more  x-rays  for  backache. 2  Is  this  overutilization,  or  is  it  higher  quality  of  care? 

The  remainder  of  Table  9  controls  for  definite  first  visit  versus  definite  return  visit.3  When 
we  look  at  the  differences  in  ordering  rates  for  first  visits  and  return  visits,  we  observe  that 
for  "x-ray  conditions”  in  general,  the  ordering  rate  is  higher  for  first  visits  than  for  return 
visits.  The  significantly  greater  number  of  x-rays  ordered  on  first  visits  for  "x-ray  conditions” 
by  PAs  and  PCNPs  does  not  extend  to  return  visits,  where  there  is  no  significant  difference. 

Despite  the  small  number  of  observations,  it  is  of  interest  that  all  three  provider  groups 
decrease  their  x-ray  ordering  rate  for  headaches  on  return  visits.  In  the  case  of  ischemic  and 
other  heart  diseases,  the  physicians  initially  have  a  lower  ordering  rate,  and  their  rate  in¬ 
creases  on  return  visits,  while  the  PEs’  ordering  rates  decline.  In  the  case  of  sinusitis,  for  all 
provider  groups,  there  is  no  trend  toward  a  decreased  ordering  rate  on  return  visits,  which 
suggests  that  providers  may  become  more  concerned  with  some  patients  who  have  not  pro¬ 
gressed  satisfactorily  by  the  time  of  a  return  visit. 

PCNPs  show  a  marked  and  significantly  higher  rate  of  ordering  x-rays  for  backache,  a 
difference  which  is  limited  to  the  first  visit.  For  backache,  the  PCNPs’  ordering  rate  of  more 
than  42  x-rays  per  100  visits  on  first  visit  is  so  high  that  one  might  well  question  whether  all 
of  these  x-rays  were  necessary.  In  the  case  of  return  visits  for  arthritis  and  joint  pain,  where 
both  the  PAs  and  PCNPs  are  ordering  x-rays  significantly  more  often  than  are  physicians,  it 
is  impossible  to  guess  if  the  patients  really  required  x-rays.  Overall  (first  and  return  visits), 
there  is  no  significant  difference  in  the  ordering  rate  of  x-rays  for  arthritis  and  joint  pain 
among  the  three  provider  groups. 


2  For  many  diagnostic  subsets  (here  in  the  x-ray  section,  and  in  following  sections  reporting  other 
tests),  numbers  of  cases  are  so  small  that  it  is  difficult  to  demonstrate  statistically  significant  differences. 
Yet,  it  is  still  possible  to  detect  a  clear  pattern  of  higher  utilization  of  x-rays  (for  appropriate  conditions) 
by  PEs— a  pattern  supported  by  the  statistically  significant  differences  found  in  the  aggregate  "x-ray  con¬ 
ditions”  category. 

3The  total  number  of  visits  is  lower  than  in  previous  x-ray  data  because  we  have  discarded  all  visits  with  ambiguous  or 
unindicated  first  or  return  status.  We  include  only  those  visits  which  we  are  certain  are  first  visits  or  return  visits. 


Table  9 

X-Rays  per  100  Visits 


Condition _ MD _ PA 

First  and  Return  Visits 


Statistically 

Significant? 


"X-ray  conditions" 

17.1 

n=t>84 

24.? 

n-1419 

22.4 

n-317 

yes 

headaches 

4.3 

n*46 

12.7 

n=102 

16.7 

n*24 

no 

Ischemic  and  other  heart 

diseases 

14.4 

n-97 

18.8 

n-32 

7.7 

n-13 

no 

Sinusitis 

14.3 

n*49 

27.6 

n-116 

24.0 

n-25 

no 

Backache 

8.6 

n-58 

14.1 

n=185 

27.6 

n*58 

yes 

Arthritis  and  joint  pain 

21.5 

n=79 

29.2 

n-240 

30.0 
n*4  0 

no 

First  Visits  Only 

"X-ray  conditions*' 

21.0 

n-271 

31.6 

n-686 

30.5 

n*lb4 

yes 

Headaches 

10.5 

n-19 

19.5 

n-41 

33.3 

n-9 

no 

Ischemic  and  other  heart 

diseases 

0.3 

n-16 

27.3 

n*ll 

33.3 

n-3 

no 

Sinusitis 

5.9 

n=l7 

25.0 

n=*56 

27.8 

n-18 

no 

Backache 

3.4 

n=29 

13.3 

n-75 

42.9 

n-28 

yes 

Arthritis  and  joint  pain 

48.0 

n-25 

42.9 

n*112 

35.3 

n-17 

no 

Return  Visits  Only 

"X-ray  conditions" 

12.1 

n=*224 

15.8 

n=524 

15.1 

n=126 

no 

Headaches 

0 

n*16 

7.1 

n»42 

10.0 

n«10 

— 

Ischemic  and  other  heart 

diseases 

15.7 

n*51 

13.3 

n*15 

0 

n*8 

no 

Sinusitis 

20.0 

n-10 

29.2 

n*43 

20.0 

n*5 

no 

Backache 

18.8 

n*16 

14.8 

n*88 

n.i 

n-27 

no 

Arthritis  and  joint  pain 

3.2 

n-31 

20.7 

n-92 

26.1 

n*23 

yes 
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ORDERING  OF  PHYSICAL  THERAPY 

Turning  now  to  a  procedure  (physical  therapy.  Table  10),  we  note  that  the  two  PE  groups 
order  more  physical  therapy  for  a  group  of  conditions  where  physical  therapy  might  well  be 
helpful.  The  difference  is  statistically  significant;  physicians  are  ordering  far  less  physical 
therapy  for  those  conditions.  The  three  subgroups  selected  for  physical  therapy — physical 
therapy  group  (excluding  fractures,  sprains,  and  strains),  arthritis  and  joint  pain,  and  back¬ 
ache — also  show  higher  ordering  rates  for  PAs  and  PCNPs.  These  differences  persist  when 
return  visits  are  viewed  separately. 

However,  it  seems  clear  that  PEs  are  not  overloading  the  physical  therapy  department. 
Referring  back  to  Table  8,  note  that  the  difference  in  the  rate  of  ordering  physical  therapy 
per  100  visits  for  all  conditions  amounts  to  fewer  than  two  procedures  ordered  per  100  visits. 
It  is  possible,  however,  that  the  physicians  are  not  ordering  as  much  physical  therapy  as  might 
be  consistent  with  good  quality  (for  example,  no  physical  therapy  ordered  for  patients  with 
arthritis  and  joint  pain  or  with  backache).  If  there  were  reason  to  suspect  that  this  were  the 
case,  it  would  be  possible  to  review  a  group  of  charts  bearing  specific  diagnoses  to  decide  if 
a  problem  existed. 


ORDERING  OF  ELECTROCARDIOGRAMS 

Table  11  presents  more  detailed  consideration  of  electrocardiograms  per  100  visits.  There 
are  no  statistically  significant  differences  in  this  table.  For  the  electrocardiogram  group  as  a 
whole  (first  and  return  visits),  physicians  are  ordering  more  tests  than  are  the  PEs.  Reference 
to  the  lower  parts  of  the  table  shows  that  this  difference  is  accounted  for  mainly  by  ordering 
activity  during  first  visits.  The  single  subset  entitled  "ischemic  and  other  heart  diseases” 
contains  too  small  a  number  of  visits  handled  by  PAs  and  PCNPs  to  draw  any  conclusions  with 
confidence.  It  is  certainly  reasonable  to  conclude  that  electrocardiograms  are  not  be¬ 
ing  overordered  by  PAs  and  PCNPs. 


ORDERING  OF  URINALYSES  AND  OTHER  TESTS 

Table  12  shows  the  differences  in  rates  of  ordering  urinalysis  per  100  visits.  There  are  no 
significant  differences,  whether  the  analysis  is  limited  to  urinary  tract  infection,  or  to  a 
somewhat  broader  category  of  urinary  tract  infection  along  with  urethritis  and  other  urinary 
tract  diseases.  Physicians  evidently  see  a  smaller  number  of  these  cases  than  do  PEs.  This 
accounts  for  the  statistically  significant  difference  in  ordering  rates  that  appeared  in  Table  8. 

The  final  two  test  categories  listed  in  Table  8  were  not  analyzed  by  subsets  of  conditions, 
or  by  first  and  return  visits.  For  all  provider  groups,  the  differences  in  ordering  rates  for  these 
tests  per  100  visits  for  all  conditions  were  not  statistically  significant. 


RATES  OF  ORDERING  TESTS:  APPLICATION 

Even  though  it  is  difficult  to  determine  whether  the  differences  are  attributable  to  varia¬ 
tions  in  quality  of  care  or  patterns  of  utilization,  utilization  analysis  is  important  because  it 
signals  potential  problem  areas.  If  desired,  it  would  be  possible  to  follow  up,  by  means  of  chart 
review,  any  significant  or  inexplicable  differences  discovered.  This  method  can  therefore  serve 
as  a  flagging  device  for  uncovering  areas  where  chart  review  might  permit  more  certain 
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Table  10 

Physical  Therapy  per  100  Visits 
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Condition 

MD 

PA 

PCNP 

Statistically 

Significant? 

First  and  Return  Visits 

"Physical  therapy  group" 

1.0 

n=384 

3.1 

n=963 

9.8 

n*204 

yes 

FT  group  excluding 
fracture,  sprain,  strain 

0 

n=160 

9.3 

n-525 

12.1 

n*116 

yes 

Arthritis  and  joint  pain 

0 

n-79 

5.0 

n-240 

5.0 

n=40 

no 

Backache 

0 

n=58 

14.1 

n=l85 

15.6 

n=58 

yes 

First 

Visits  Only 

"Physical  therapy  group" 

0.6 

n*176 

9.  7 
n=485 

11.3 

0*106 

yes 

PT  group  excluding 
fracture,  sprain,  strain 

0 

n*64 

9.0 

n*i244 

13.8 

n=58 

yes 

Arthritis  and  joint  pain 

0 

n=25 

5.4 

n*112 

5.9 

n=17 

-- 

Backache 

0 

n=29 

13.3 

n»75 

14.3 

n*28 

no 

Return 

Visits  Only 

"Physical  therapy  group" 

1.0 
n =102 

9.3 

n*335 

8.4 
n»  83 

yes 

PT  group  excluding 
fracture,  sprain,  strain 

0 

n*54 

10.5 

n*210 

10.9 

n=55 

yes 

Arthritis  and  joint  pain 

0 

n*31 

5.4 

n*92 

4.3 

n*23 

— 

Backache 

0 

n*16 

14.8 

n*88 

18.5 

n-27 

no 

d 
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Table  11 


Electrocardiograms  per  100  Visits 


Condition 

MD 

PA 

PCNP 

Statistically 

Significant? 

First  and  Return 

Visits 

"Electrocardiogram  group"  12.5 

n*289 

8.0 

n-511 

8.5 

n-117 

no 

Ischemic  and  other  heart 

diseases  25.8 
n*9  7 

21.9 

n-32 

7.7 

n=l3 

no 

First  Visits  Only 

"Electrocardiogram  group' 

21.1 

n-38 

12.5 

n=96 

15.0 

n=20 

no 

Ischemic  and  other  heart 

diseases  18.8 
n=16 

27.3 

n-11 

33.3 

n=3 

no 

Return  Visits  Only 

"Electrocardiogram  group' 

8.9 

n=191 

7.5 

n-333 

7.7 

n-78 

no 

Ischemic  and  other  heart 

diseases  25.5 
n-51 

13.3 

n=15 

0 

n=8 

_ 

Table  12 


Urinalyses  per  100  Visits 


Condition 

MD 

PA 

PC1IP 

Statistically 

Significant? 

First  Visits  Only 

Urinary  tract  Infection  (UTI) 

60.0 

n-45 

63.9 

n-155 

80.0 

n*40 

no 

UTI,  urethritis,  and  other 
urinary  tract  diseases 

43.9 

n«82 

54.9 

n=255 

60.9 

n=69 

no 

First 

Visits  Only 

UTI 

84.2 

n-19 

71.2 

n=69 

95.0 

n-20 

no 

UTI,  urethritis,  and  other 
urinary  tract  diseases 

55.6 

n-36 

61.8 

n=110 

71.0 

n-31 

no 

Return 

Visits  Only 

UTI 

37.5 

n=8 

58.0 

n«69 

58.8 

n-17 

no 

UTI,  urethritis,  and  other 
urinary  tract  diseases 

40.0 

n*2u 

52.1 

n-117 

47.1 

n=34 

no 

conclusions  to  be  drawn  concerning  the  presence  of  high  or  low  quality,  or  too  high  or  too  low 
utilization  of  services.4 


DISPOSITION  IN  GENERAL 

In  addition  to  the  possible  quality-of-care  implications  of  overordering  tests  and  proce¬ 
dures,  such  overordering  (which  has  not  been  demonstrated  here)  would  stress  finances  and 
personnel.  A  system  could  also  be  stressed  by  "overordering”  of  return  visits  to  clinics.  In 
medical  jargon,  the  phrase  "disposition  of  visit”  refers  to  the  provider’s  decision  at  the  end 
of  every  visit  on  what  will  be  the  subsequent  contact,  if  any,  with  the  patient.  We  will  review 
data  for  the  end  of  first  visits  alone,  then  return  visits  alone,  and  finally  for  all  visits  combined. 
All  tables  show  the  disposition  categories  used  by  the  National  Ambulatory  Medical  Care 
Survey  in  its  publications.  The  different  possibilities  for  disposition  range  from  no  expected 
future  contact  for  this  problem,  through  telephone  contact,  to  definite  reappointment  or 
referral.  Admission  to  the  hospital  is  also  a  possible  disposition. 

It  is  worthwhile  to  compare  the  dispositional  behavior  of  various  groups  of  providers.  If 
one  group  orders  many  more  return  visits  or  referrals  for  its  patient  population  than  does 
another  group  with  a  similar  patient  population,  it  is  likely  that  either  one  group  is  overorder¬ 
ing  or  the  other  is  underordering.  Either  possibility  might  reflect  differences  in  the  quality 
of  care  delivered;  in  addition,  "extra”  visits  would,  in  and  of  themselves,  stress  the  medical 
care  delivery  system,  and  would  likely  generate  additional  tests  and  procedures.  Do  our  data 
in  fact  show  overburdening  or  underutilization  of  the  system  by  one  or  another  type  of 
provider?  What  happened  in  a  situation  in  which  the  PEs  had  great  freedom  in  deciding 
whether  or  not  to  make  return  appointments? 

Table  13  provides  this  information  for  first  visits  only  and  for  return  visits  only,  while 
Table  14  provides  it  for  all  visits  combined.  Both  types  of  visits,  of  course,  necessitate  decisions 
regarding  follow-up.5 

At  the  end  of  first  visits,  where,  in  general,  triage  was  purposely  not  carried  out,  we  see 
that  disposition  patterns  are  very  similar  among  the  three  provider  types.  The  "no  follow-up” 
category  reflects  the  acute  and  uncomplicated  nature  of  many  problems  treated  in  the  primary 
medical  setting.  There  is  little  "telephone  follow-up”  by  any  type  of  provider.  Definite  return 
appointments  are  shown  in  the  row  labeled  "return  at  specified  time.”  These  rates  of  definite 
return  are  virtually  identical,  which  is  not  surprising  given  the  similarity  in  patient  population 
seen  on  first  visits.  This  result  provides  satisfying  support  for  the  position  that  PAs  and  PCNPs 
are  neither  overordering  nor  underordering  visits.  This  row  also  furnishes  evidence  that 
neither  PAs  nor  PCNPs  are  generating  an  untoward  number  of  return  visits  in  comparison 
with  physicians.  Referral  patterns  are  virtually  identical  as  well.6 

Testing  for  statistical  significance  using  a  Chi-square  test,  we  find  that  the  physician 
distribution  of  dispositions  is  significantly  different  from  both  the  PAs’  distribution  (X2  = 
78.47, 5  d.f.)  and  the  PCNPs’  distribution  (X2  =  51.71, 5  d.fi).  Because  each  category  of  disposi¬ 
tion  contributes  to  the  total  Chi-square  statistic,  the  larger  the  contribution  of  any  category 


4The  reader  is  invited  to  review  the  results  of  the  other  method  we  presented  for  investigating  possible  inappropriate 
utilisation  ti  e.,  selecting  specific  tests,  procedures,  or  prescriptions  and  analyzing  whether  or  not  they  were  employed  in 
situations  likely  to  have  been  inappropriate).  See  App.  C,  "Discussion  of  Selected  Criteria  of  Quality  .” 

5  All  three  tables  include  all  physicians  employed  in  primary  care  settings,  not  only  those  who  are  team  members. 

"A  priori,  one  might  expect  a  higher  return  and  referral  rate  from  PEs  at  the  end  of  first  visits,  because  certain  patients 
who  really  should  have  been  seen  by  a  physician  would  have  been  "mismatched"  to  a  PA  or  PCNP  and  would  need  to  be 
reassigned  to  a  physician  for  an  additional  visit.  Our  data  do  not  demonstrate  this  pattern,  probably  because  that  type  of 
patient  usually  has  a  problem  that  a  physician  could  not  deal  with  in  a  single  visit,  either.  Therefore,  the  physician  would 
also  call  for  a  definite  return  or  referral 


26 


Table  13 

Disposition  of  Visits  in  Primary  Medical  Settings: 
Demonstration  Bases,  1977 


Disposition 

MD  Only 

PA  Only0 

PCUP  Only0 

First 

Visits 

No  follow-up /re turn  if  needed 

58% 

62% 

63% 

Telephone  follow-up  ^ 

2 

<1 

2 

Return  at  specified  time 

25 

27 

25 

Refer  to  other  MD 

9 

8 

10 

Return  to  referring  MD 

<1 

<1 

<1 

Admit  to  hospital 

5 

2 

<1 

Total  *  5383  visits 

n=1449 

n-3518 

n=4l6 

HeUam  Visits 

(a) 

(c) 

No  follow-up/ return  if  needed 

33  X 

47% 

51% 

Telephone  follow-up  ^ 

2 

<1 

2 

Return  at  specified  time 

54 

41 

34 

Refer  to  other  MD 

9 

10 

12 

Return  to  referring  MD 

<1 

<1 

0 

Admit  to  hospital 

2 

1 

<1 

Total  =»  4679  visits 

n=1399 

n=2589 

n=691 

aPA  and  PCilP  distributions  are  statistically  significantly 
different  from  MD  distributions  (p  <  0.005)  using  Chi-square 
tests  (PA  X2  *  78.47,  PCNP  X2  -  51.71,  each  with  5  d.f.). 

^Includes  1.0  percent  'Admit  to  Quarters." 

^Again  statistically  different  (p  <  0.005,  PA  X2  =  95.81, 
PCNP  X2  -  96.11) 


Table  14 


Disposition  of  Visits  in  Primary  Medical  Settings, 
Demonstration  Bases,  1977:  First  and 
Return  Visits  Combined 


Disposition 

MD 

Only 

PA 

Only*2 

PCNP 

0nlya 

Combined 

AF  Providers 

GP/FP, 

IIAMCS^ 

No  follow-up/retum  if  needed 

48% 

57% 

58% 

54% 

45% 

Telephone  follow— jp 

2 

<1 

2 

2 

4 

Return  at  specified  time 

35 

32 

29 

32 

51 

Refer  to  other  MD 

9 

9 

11 

9 

3 

Return  to  referring  MD 

<1 

<1 

<1 

<1 

1 

Admit  to  hospital 

4 

2 

<1 

2 

1 

Total  *  14063  visits  n- 

■3810 

n-7U95 

ne193b 

^PA  and  PCNP  distributions  are  statistically  signif icantly  different 
from  the  MD  distribution  (p  <  0.005)  using  Chi-square  tests  (PA  = 
140.57,  PCNP  X2  =  98.16,  each  with  5  d.f.). 

J, 

1975  data  (Advance  data,  No.  15,  December  14,  1977,  p.  9). 

°Zncludes  1.0  percent  ’’Admit  to  Quarters." 


to  the  statistic,  the  more  that  category  helps  to  cause  the  conclusion  that  differences  are 
significant.  The  difference  between  MDs  and  PAs  in  the  use  of  telephone  follow-up  accounted 
for  much  of  the  significance  of  the  Chi-square  statistic.  The  differences  in  rates  of  hospitaliza¬ 
tion  between  physicians  and  both  PAs  and  PCNPs  also  accounted  for  a  large  portion  of  the 
total  difference. 

An  interesting  finding  is  the  distinctly  higher  hospitalization  rate  by  physicians  at  the  end 
of  first  visits,  and  the  difference  is  maintained  (though  to  a  lesser  extent)  at  the  end  of  return 
visits.  This  finding  suggests  the  presence  of  an  informal  triage  system  whereby  patients  who 
appear  to  be  seriously  ill  are  identified  and  steered  to  the  physician  even  on  the  first  visit.  In 
fact,  we  observed  such  systems  in  action  at  the  front  desks  of  the  teams. 

Turning  to  return  visits,  we  again  note  evidence  of  similar  dispositional  patterns  and 
similar  ordering  of  return  visits  on  the  part  of  all  three  provider  groups.  As  expected,  fewer 
of  the  patients  who  already  have  been  told  to  come  back  for  one  return  visit  are  told  that  no 
further  follow-up  will  be  needed.  To  the  contrary,  the  row  labeled  "return  at  specified  time” — 
that  is,  definite  return  appointments — is  higher  for  all  three  groups  on  return  visits  as  opposed 
to  first  visits.  This  change  is  consistent  with  the  notion  that  a  number  of  patients  coming  for 
return  visits  have  chronic  conditions  that  will  require  regular  contact.  In  our  opinion,  the 
observation  that  physicians  showed  the  highest  rate  of  ordering  definite  return  appointments, 
and  the  lowest  rate  for  deciding  that  no  follow-up  is  needed,  does  not  imply  that  MDs  overorder 
return  visits  or  that  PEs  underorder;  rather,  we  believe  it  reflects  the  difference  in  patient  mix 
on  return  visits,  when  physicians  are  seeing  a  higher  proportion  of  the  more  serious  and  more 
chronic  cases.7 

At  the  end  of  return  visits,  PAs  and  PCNPs  both  show  insignificantly  higher  rates  of 
referral.  Allowing  for  the  likelihood  that  the  seriousness8  of  cases  seen  on  return  visits  by  PAs 
and  PCNPs  will  be  somewhat  lower  than  the  seriousness  of  cases  seen  by  physicians,  we 
conclude  that  when  case  mix  is  considered,  PAs  and  PCNPs  do  generate  slightly  higher 
referral  rates  than  do  physicians.  We  do  not  find  these  referral  rates  worrisome.  Had  they 
been  much  higher  than  that  of  the  physicians,  we  might  have  suspected  overburdening  of  the 
system;  at  this  rate  of  referral,  however,  it  is  most  reasonable  to  conclude  that  the  less  highly 
trained  providers  are  simply  more  cautious.  There  is  certainly  no  gross  imposition  of  visits  on 
the  system  because  of  this  extra  caution. 

Again,  tests  for  statistical  significance  show  differences  between  MDs’  distributions  and 
that  of  both  PAs  and  PCNPs  (PA  X2  =  95.81,  PCNP  X2  =  96.11  with  5  d.f.).  Greater  frequency 
of  return  visits  on  the  part  of  physicians  and  greater  frequency  of  "no  definite  follow-up”  on 
the  part  of  PAs  and  PCNPs  contributed  the  main  portion  of  the  significant  Chi-square  statistic. 
Lower  PCNP  rates  for  hospitalization  were  also  important. 

Table  14  above  combines  first  and  return  visits.  We  include  it  to  enable  a  comparison  with 
the  civilian  sector.  For  purposes  of  comparison,  we  have  combined  all  three  Air  Force  provider 
groups.  We  note  a  higher  rate  of  scheduled  return  appointments  ("return  at  specified  time”) 
in  the  civilian  sector  (GP/FP  NAMCS  column).  The  difference  might  be  attributable  to  differ¬ 
ences  in  age  distribution  or  complexity  of  problem  in  the  two  settings.  There  are  more  internal 
referrals  within  the  Air  Force  system.  However,  when  we  add  the  Air  Force’s  internal  refer¬ 
rals  to  its  definite  return  appointments,  we  still  note  more  scheduled  returns  in  the  civilian 
sector.  To  speculate  on  the  reasons  for  these  differences  would  go  beyond  what  our  data 
permit. 


7Several  criteria  presented  in  Sec.  Ill  in  fact  showed  that  physicians  tend  to  order  return  visits  more  often  for  a  group 
of  conditions  that  would  probably  warrant  a  return  visit. 

*See  Sec.  V  for  further  discussion  of  seriousness. 
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CONCLUSIONS 

To  summarize  our  analysis  of  utilization,  let  us  review  the  principal  findings. 

First,  the  condition-specific  criteria  in  Sec.  Ill,  which  dealt  with  possible  unwarranted 
ordering  of  tests,  procedures,  or  therapies  for  specific  conditions  (the  so-called  "negative 
criteria”)  failed  to  demonstrate  unnecessary  utilization  on  the  part  of  PAs  and  PCNPs  as 
compared  with  physicians. 

Second,  our  analysis  of  ordering  rates  for  selected  tests  and  procedures  did  not  give 
evidence  of  misordering  on  the  part  of  these  two  types  of  PEs. 

Third,  the  documentation  of  disposition  patterns  shows  that,  with  regard  to  internal 
generation  of  clinic  utilization,  all  three  provider  groups  behaved  similarly.  In  short,  there  i3 
no  evidence  for  overordering  or  overutilization  on  the  part  of  PAs  and  PCNPs. 


V.  SUPERVISION  OF  PHYSICIAN’S 
EXTENDERS 


Both  physician’s  assistants  (PAs)  and  primary  care  nurse  practitioners  (PCNPs)  are 
trained  to  work  under  the  supervision  of  a  physician.  In  a  system  such  as  that  used  at  the 
demonstration  bases,  without  structured  triage,  a  physician  must  be  available  to  supervise  the 
extenders’  handling  of  complex  problems  and  to  assume  responsibility  for  the  most  serious 
cases.  The  availability  of  supervision  does  not  in  itself  guarantee  that  the  supervision  is 
effective  or  that  it  improves  the  quality  of  care  given,  but  the  absence  of  the  supervisory 
channel  or  lack  of  evidence  of  its  use  would  signal  potential  problems. 

While  too  little  contact  between  extenders  and  supervisors  could  cause  concern,  too  much 
supervision  could  also  signal  a  problem.  PAs  and  PCNPs  are  trained  to  handle  a  large  number 
of  routine  problems  in  an  outpatient  setting  without  a  physician’s  direct  involvement. 
Supervision  of  an  excessively  large  number  of  visits  wastes  physicians’  time  and  suggests 
poor  extender  performance. 

To  investigate  the  adequacy  of  supervision  during  the  demonstration  project,  we  again 
analyzed  the  Patient  Contact  Record  data.  We  wanted  to  know  to  what  extent  a  physician  was 
involved  in  the  care  of  patients  seen  by  an  extender. 

Several  levels  of  involvement  are  possible.  The  extender  may  see  that  a  patient’s  problems 
require  a  physician’s  expertise,  and  so  transfer  the  patient  to  the  supervising  physician,  or 
refer  the  patient  to  a  specialist  within  the  base  hospital.  The  extender  may  consult  the 
supervising  physician,  who  will  decide  whether  or  not  to  speak  with  or  examine  the  patient 
before  advising  the  extender.  Finally,  the  extender  may  need  only  a  countersignature  for 
prescriptions,  referrals  outside  the  hospital,  or  other  specified  orders  or  dispositions. 

Even  when  the  visit  itself  does  not  include  any  physician’s  involvement,  a  return  visit  by 
the  patient  might.  Patients  told  to  return  to  the  clinic  might  be  scheduled  to  see  the  physician 
team  member  on  their  return  visit.  Any  patient  admitted  to  the  hospital  will  be  reviewed  by 
a  physician. 

Physician  involvement  ought  to  be  more  likely  in  cases  where  the  patient  has  a  problem 
that  is  complex  or  has  serious  potential  consequences.  It  also  ought  to  be  more  likely  to  occur 
(for  a  wider  range  of  problems)  if  the  PE  believes  the  patient’s  condition  to  be  serious  or  very 
serious. 

We  have  divided  visits  to  extenders  into  four  categories  of  varying  physician  involvement. 
The  first  category,  "consultation  with  physician,”  includes  all  visits  where  an  extender  sought 
some  assistance  from  the  supervising  physician.  Assistance  may  include  seeing  the  patient,  but 
it  does  not  include  cases  where  only  countersignature  was  sought.  The  second  category, 
"referred  to  physician,”  includes  referrals  to  other  clinics  or  CHAMPUS  on  the  same  day  or 
for  a  future  appointment,  and  also  includes  hospital  admissions.  The  third  category,  "to 
return,”  includes  all  visits  when  the  PE  handled  the  case  without  a  physician’s  involvement 
and  when  the  PEs  instructed  the  patient  to  make  a  return  appointment,  as  well  as  visits  when 
the  patient  was  admitted  to  quarters.  Since  we  cannot  determine  whether  a  physician  or  an 
extender  saw  the  patient  on  the  return  visit,  this  category  represents  only  the  possibility  of 
physician  involvement.  Finally,  a  fourth  category  includes  visits  when  no  physician  was 
involved  and  when  the  patient  was  instructed  to  return  or  telephone  if  needed,  or  when  no 
formal  follow-up  was  planned. 

We  further  divided  the  conditions  listed  on  the  Patient  Contact  Record  into  three  groups: 
1)  physician  generally  not  needed;  2)  physician  may  or  may  not  be  needed;  and  3)  physician 


29 


30 


generally  needed.  The  first  group  included  conditions  where  the  PEs  could  treat  the  typical 
case  of  that  condition  without  the  need  for  direct  physician  involvement.  While  there  may  be 
some  disagreement  about  what  conditions  extenders  should  be  treating,  this  group  includes 
conditions  about  which  there  is  little  question.  The  second  group  includes  those  conditions 
about  which  there  might  be  a  question:  Some  observers  would  be  comfortable  allowing  extend¬ 
ers  to  treat  these  conditions  without  direct  physician  involvement,  while  others  would  prefer 
that  the  physician  handle  these  conditions.  Finally,  the  third  group  includes  conditions  where 
most  would  agree  that  a  physician  is  needed.1  This  group  comprises  the  few,  most  serious, 
conditions,  e.g.,  malignant  neoplasms  (cancers)  or  arrhythmias  (irregularities  of  heartbeat). 
For  all  three  groups,  the  division  was  made  on  the  basis  of  the  diagnostic  labels  alone,  with 
necessary  disregard  for  particulars  of  the  disease,  or  of  the  individual’s  specific  medical  status. 
Thus,  for  some  "MD  not  needed”  conditions,  a  consultation  might  well  be  appropriate,  in 
certain  circumstances.  Similarly,  some  patients  with  conditions  (e.g.,  chronic,  stable  atrial 
fibrillation — an  arrhythmia)  that  must  be  designated  as  "MD  needed”  because  of  their 
diagnostic  labels  can  in  fact  be  appropriately  managed  by  an  extender  without  need  for  direct 
physician  involvement. 

Table  15  shows  the  extent  of  physician  involvement  in  visits  seen  by  PAs.  The  proportion 
of  visits  in  which  a  physician  actually  participated  is  shown  for  each  group  of  conditions.  First 
visits  and  return  visits  are  separated.  (Where  more  than  one  condition  is  indicated  for  a  visit, 
the  condition  most  likely  to  need  physician  involvement  is  the  one  identified  with  the  visit.) 

Table  15  shows  that  the  bulk  of  first  visits  seen  by  the  PAs  are  for  conditions  usually  not 
requiring  physician  involvement.  For  these  visits,  PAs  rarely  seek  a  physician’s  advice  in 
handling  the  case.  Only  3  percent  of  these  cases  involve  a  physician  in  the  care,  and  only  8 
percent  are  referred  to  a  physician.  Conditions  that  are  more  likely  to  require  physician 
involvement  are  fewer  in  number.  For  these  cases,  PAs  are  much  more  likely  to  consult  with 
a  physician  about  the  case  (18  percent  when  "usually  needed”  and  12  percent  when  "may  be 
needed”).  PAs  are  also  much  more  likely  to  refer  these  cases  to  a  physician  (22  percent  when 
"usually  needed”  and  14  percent  when  "may  be  needed”).  For  return  visits,  the  results  are 
quite  similar.  Table  16  shows  the  same  statistics  for  PCNPs.  The  level  of  physician  involvement 
for  PCNPs  is  similar  to  that  for  PAs. 

Tables  17  and  18  show  consultation  rates  for  visits  divided  according  to  the  extenders' 
evaluation  of  the  seriousness  of  the  problem.  Again,  we  see  a  greater  likelihood  of  consulting 
a  physician  for  serious  problems.  For  PAs  the  likelihood  of  seeking  consultation  increases 
twelve-fold  between  "not  serious”  and  "very  serious/serious”  cases.  PCNPs  also  are  much 
more  likely  to  seek  a  physician’s  advice  on  serious  or  very  serious  cases. 

It  is  difficult  to  assess  the  appropriateness  of  the  level  of  physician  supervision  and 
consultation  that  we  see  from  these  data.  The  rates  of  5  percent  for  first  visits  and  8  percent 
for  return  visits  do  not  put  any  serious  burden  on  physicians’  time.  But  is  this  level  of  physician 
supervision  enough  to  ensure  high-quality  PE  performance? 

One  standard  for  comparison  comes  from  a  recent  evaluation  study  of  PAs’  performance 
in  a  prepaid  health  plan  setting  (Kaiser-Permanente  Program,  Portland,  Oregon),  a  setting 
similar  to  the  Air  Force  clinics  in  the  demonstration  (Record  et  al.,  1977a).  The  study  reviewed 
charts  of  patients  who  had  seen  PAs,  and  recorded  the  frequency  and  appropriateness  of 
consultation  in  about  12  percent  of  the  cases.  These  cases,  however,  included  an  unspecified 
number  of  requests  for  countersignature,  a  type  of  consultation  excluded  from  our  results  for 
the  Air  Force  clinics.  A  separate  sample  of  consultations  in  the  same  clinic  showed  that  31  out 
of  83  consultations  were  for  physician  signature  on  forms  or  prescriptions.  Applying  this  rate 
to  the  12  percent  average  gives  a  rate  of  7  to  8  percent  for  consultations  excluding  those  for 


’Appendix  E  lists  the  diagnoses  specific  to  each  group. 
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Table  15 

PAs'  use  of  Physician  Consultation,  Referral,  and  Return 
Appointments,  by  Main  Condition’s  Need  for  a  Physician 


MD 

MD 

M0 

Extent  of 

Usually 

May  Be 

Usually 

All 

MD  Involvement 

Needed 

Needed 

Not  Needed 

Visits 

First  Visits 

Consultation 

18% 

124 

34 

5% 

Referred  to  MD 

22 

14 

8 

9 

To  return 

28 

32 

24 

25 

No  definite  follow-up 

32 

42 

65 

61 

Total 

1007. 

100% 

1004 

1004 

n=40 

n=628 

n=2961 

n=3629 

he  turn  \ 

'isits 

Consultation 

16% 

10% 

74 

84 

Referred  to  MD 

14 

9 

9 

9 

To  return 

44 

53 

31 

36 

No  definite  follow-up 

26 

28 

52 

47 

Total 

1004 

1004 

100  % 

1004 

n=43 

n-516 

n=2188 

n=2747 

NOTES;  An  independent  judge  determined  the  need  for  a 
physician,  based  on  main  condition.  Cases  were  eliminated 
if  they  provided  incomplete  data  on  the  particular  questions 
used  for  this  table. 


Table  16 

PCNPs’  use  of  Physician  Consultation,  Referral,  and  Return 
Appointments,  by  Main  Condition’s  Need  for  a  Physician 


Extent  of 

MD  Involvement 

MD 

Usually 

Needed 

MD 

May  Be 
Needed 

MD 

Usually 
Not  Needed 

All 

Visits 

First  Vtsits 

Consultation 

114 

134 

54 

74 

Referred  to  MD 

33 

13 

6 

8 

To  return 

33 

30 

21 

22 

No  definite  follow-up 

22 

44 

68 

63 

Total 

1004 

1004 

1004 

1004 

n*18 

n-179 

n=853 

n»1050 

he  turn 

Visits 

Consultation 

04 

94 

9% 

94 

Referred  to  MD 

50 

10 

9 

10 

To  return 

30 

48 

26 

30 

No  definite  follow-up 

20 

33 

55 

51 

Total 

1004 

1004 

1004 

1004 

n»10 

n-128 

n-553 

n-691 

UOTES :  An  independent  judge 

determined  the  need 

for  a 

physician,  based  on  main  condition.  Cases  were  eliminated 
if  they  provided  Incomplete  data  on  the  particular  questions 
used  for  this  table. 
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Table  17 

PAs’  use  of  Physician  Consultation,  Referral,  and  Return 
Appointments,  by  Seriousness  of  Condition  Treated 


Extent  of 

Very  Serious/ 

Slightly 

ilot 

HD  Involvement 

Serious 

Serious 

Serious 

First  Visits 


Consultation 

36% 

10% 

3% 

Referred  to  MD 

17 

16 

7 

To  return 

35 

49 

19 

Wo  definite  follow-up 

12 

25 

71 

Total 

100% 

100  % 

100% 

n=83 

n=650 

n=2974 

Return  Visits 


Consultation 

32% 

12% 

4% 

Referred  to  MD 

10 

14 

7 

To  return 

51 

53 

28 

Ho  definite*  follow-up 

_ 6 

21 

60 

lotal 

100% 

100% 

100% 

n=125 

n=689 

n=1935 

NOTES:  The  practitioner  determined  seriousness. 
Cases  were  eliminated  if  they  provided  incomplete  data 
on  the  particular  questions  used  for  this  table. 


Table  18 

PCNPs’  use  of  Physician  Consultation,  Referral,  and  Return 
Appointments,  by  Seriousness  of  Condition  Treated 


Extent  of 

Very  Serious/ 

Slightly 

Not 

MD  Involvement 

Serious 

Serious 

Serious 

First  Visits 

Consultation 

22% 

14% 

4% 

Referred  to  MD 

13 

11 

7 

To  return 

35 

35 

19 

No  definite  follow-up 

30 

40 

69 

Total 

100% 

100% 

100% 

CM 

II 

e 

n-186 

n-853 

Return  Visits 


Consultation 

11% 

10% 

8% 

Referred  to  MD 

11 

11 

7 

To  return 

42 

4b 

25 

lio  definite  follow-up 

37 

34 

60 

Total 

100% 

100% 

100% 

n-38 

n-167 

n-463 

NOTES:  The  practitioner  determined  seriousness. 
Cases  were  eliminated  if  they  provided  incomplete  data 
on  the  particular  questions  used  for  this  table. 
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a  signature  only.  Further,  the  civilian  clinic  where  the  PAs  were  practicing  was  constrained 
by  law  to  structure  physician  involvement  in  the  treatment  process.  Chart  reviews  showed 
that  PAs  erred  on  the  side  of  consultation  when  not  needed,  rather  than  erring  by  failing  to 
consult  when  needed.  Considering  these  differences,  the  5  to  8  percent  consultation  rate  at  the 
demonstration  bases  compares  closely  with  this  civilian  experience. 

Another  indication  of  PEs’  performance  and  appropriateness  of  supervision  comes  from 
hospital  commanders  and  supervising  physicians  at  the  demonstration  bases.  During  the 
course  of  the  demonstration,  no  incidents  were  reported  that  indicated  that  PEs  were  failing 
to  consult  when  necessary  with  their  supervisors.  Even  as  the  extenders  at  these  bases  as¬ 
sumed  a  significant  portion  of  primary  care  delivery,  base  commanders  and  supervisors  who 
were  interviewed  expressed  confidence  in  the  extenders’  ability  to  judge  when  a  case  called 
for  the  physician’s  involvement. 

The  evidence  we  have  cited  suggests  that  the  extenders’  care  in  the  demonstration  project 
was  adequately  supervised.2  It  is  unlikely  that  a  more  detailed  study  of  supervision  behavior 
would  change  this  conclusion.  Consultation  is  an  established  part  of  the  extenders’  pattern  of 
practice.  Extenders’  consultations  occupy  a  modest  amount  of  physician  time.  The  amount  of 
consultation  does  not  differ  greatly  from  the  amount  experienced  in  a  similar  civilian  setting. 


^The  quality  of  care  information  presented  above  provides  indirect  evidence  in  support  of  this  conclusion. 


VI.  CONCLUSION 


This  report  has  presented  data  aimed  at  evaluating  the  quality  of  care  delivered  by 
physician’s  extenders  (physician’s  assistants  and  primary  care  nurse  practitioners)  who  func¬ 
tioned  in  the  primary  medical  clinics  of  Air  Force  Hospitals.  We  have  examined  the  perfor¬ 
mance  of  these  extenders  in  relation  to  the  performance  of  physicians  working  in  the  same 
settings. 

We  adduced  several  strands  of  evidence  in  evaluating  the  medical  appropriateness  of 
having  physician’s  extenders  (PEs)  assume  a  substantial  portion  of  the  workload  formerly 
carried  principally  by  physicians.  We  presented  the  results  of  condition-specific,  quality-of- 
care  criteria  applied  to  the  care  provided;  reviewed  differences  in  the  pattern  of  ordering  tests, 
procedures,  and  return  visits;  and  analyzed  the  supervisory-consultative  relationships  be¬ 
tween  physicians  and  extenders.  All  conclusions  are  based  on  data  gathered  by  encounter  form 
for  one  month,  in  the  context  of  an  ongoing  demonstration  project  at  four  Air  Force  Hospitals. 

For  the  condition-specific,  quality-of-care  criteria,  we  looked  at  five  different  classes:  desir¬ 
able  diagnostic  actions,  desirable  therapeutic  actions,  undesirable  diagnostic  actions,  undesir¬ 
able  therapeutic  actions,  and  desirable  disposition  actions.  The  summary  comparison  showed 
that,  for  a  total  of  25  out  of  28  criteria,  PAs’  performance  statistically  equalled  or  exceeded 
the  physicians’  performance;  and  PCNPs  did  the  same  for  a  total  of  14  out  of  19  criteria.  For 
most  criteria  no  significant  differences  were  found  among  the  three  groups  of  providers.  Also 
noteworthy  was  the  lack  of  a  systematic  deficiency  of  any  group  of  providers.  We  conclude 
that,  insofar  as  we  can  determine  with  our  criteria,  for  the  types  of  conditions  that  they  are 
treating  in  this  setting,  PAs  and  PCNPs  do  measure  up  to  the  performance  level  of  physicians 
working  in  the  same  setting. 

Out  of  62  criteria,  significant  differences  in  performance  between  PAs  and  PCNPs  occurred 
for  only  three  of  the  criteria.  We  therefore  conclude  that  there  is  no  overall  difference  in 
performance. 

Furthermore,  when  1977  results  were  compared  with  1974  results,  there  was  no  evidence 
of  any  worsening  in  PAs’  performance — despite  classes’  having  been  trained  at  different  times, 
a  number  of  PAs  having  been  out  of  school  for  a  longer  time  and  therefore  more  experienced 
but  potentially  less  rigorous,  and  a  higher  ratio  of  PAs  to  supervising  physicians  in  1977.  In 
addition,  based  on  the  1977  data,  we  found  that  recently  graduated  PAs  performed  equally  as 
well  as  earlier  graduates.  Therefore,  the  "product”  being  graduated  from  the  Air  Force’s 
in-house  PA  training  program  appears  to  be  both  consistent  and  stable. 

The  condition-specific  criteria  included  a  number  of  measures  of  quality  that  we  used  to 
detect  overuse  of  certain  diagnostic  and  therapeutic  actions.  The  "utilization  analysis”  section 
examined  that  possibility  by  comparing  utilization  rates  for  physicians,  PAs,  and  PCNPs 
across  a  number  of  tests  and  procedures.  Although  observed  differences  in  ordering  rates  did 
occur  among  the  three  groups  of  providers,  there  was  no  consistent  evidence  that  extenders 
significantly  overburdened  the  Air  Force’s  care  delivery  system.  Further,  with  rare  excep¬ 
tions,  it  was  difficult  to  decide  whether  observed  differences  in  ordering  rates  represented 
neglect  or  caution,  prudence  or  waste,  higher  quality  or  lower  quality. 

Likewise,  we  showed  that  PEs  are  not  generating  an  inordinate  number  of  return  visits, 
referrals,  or  hospital  admissions. 

Finally,  in  observing  the  use  of  the  supervisory-consultative  channel  established  between 
extenders  and  physicians,  we  found  that  the  extenders  involved  the  physicians  more  frequent¬ 
ly  in  complex  problems,  and  with  patients  whose  condition  was  believed  to  be  serious  or  very 
serious.  The  absolute  rate  of  consultation  between  extenders  and  physicians  was  consistent 
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with  observations  in  one  study  made  in  the  civilian  sector.  Consultation  took  up  only  a  modest 
amount  of  physicians’  time.  We  conclude  that  the  evidence  available  to  us  suggests  that 
extenders’  care  in  the  demonstration  project  was  adequately  supervised. 

In  view  of  all  these  encouraging  findings,  we  believe  that  the  Air  Force  can  deliver  the 
same  quality  of  medical  care  when  PEs  (PAs  and  PCNPs)  treat  a  sizable  proportion  of  the 
patients  formerly  treated  by  physicians,  and  that  no  quality  bar  exists  to  the  continued 
training  and  employment  of  PAs  and  PCNPs  in  Air  Force  outpatient  clinics.  We  find  the 
quality  of  care  they  deliver  to  be  acceptable  when  they  are  providing  care  for  the  types  of 
problems  they  have  been  trained  to  treat.  In  addition,  the  PAs’  performance,  as  we  have 
measured  it,  constitutes  a  strong  endorsement  of  the  Air  Force’s  in-house  PA  training  pro¬ 
gram. 


Appendix  A 

PATIENT  CONTACT  RECORD:  SAMPLE 
FORM,  AND  FREQUENCIES  OF  CONDITIONS 
MARKED  BY  SELECTED  PRACTITIONERS, 

1977 


abbeduo 


Hat 


38 


N?  307603 


1171 

ARRIVED 

EXAMINEO 

(TIME  PATIENT  IS  CALLED  TO  EXAMINATION  OR  TREATMENT  ROOM) 

18  16) 

<17  251 

(26  27)  OUTPATIENT  UNIT: 

O  »  AIR  FORCE  CLINIC 

(FAMILY  PRACTICE  OR 
GENERAL  THERAPY) 

□  2  FLIGHT  SURGEON  S  CLINIC 

□  3  PHYSICAL  EXAM  SECTION 

□  4  EMERGENCY  ROOM/WALK  IN 

CLINIC 


□  5  SICK  CALL 

□  11  OBGYN 

□ 

□  6  ALLERGY 

Q  12.  ORTHOPEDICS 

□ 

□  7  DERMATOLOGY 

□  13  PODIATRY 

a 

□  8  ENT 

□  14.  PEDIATRICS 

a 

□  9.  EYE  CLINIC 

□  15.  PSYCHIATRY 

□  10  MEDICINE 

□  16.  SURGERY 

PATIENT  CONTACT  RECORD 

APPROVED:  DBMS 

PART  I  Patient  Information 


TO  BE  FILLED  OUT  BY  PATIENT 


*|  (SPONSOR  S)  SOCIAL  SECURITY  NUMBER  (LAST  4  DIGITS) 
(33-36)  ....  - 


2  SERVICE  Of  PATIENTORPATIENT  SSPONSOR 

□  1  AIR  FORCE 
1371  02  ARMY 

□  3  NAVY 

□  4  MARINE  CORPS 

□  5  COAST  GUARD 

O  6  CADET/ APPLICANT  FOR  MILITARY  SERVICE 
O  7.  CIVILIAN  EMPLOYEE 

O  8.  OTHER  _  _ _ 


11  DID  YOU  MAKE  AN  APPOINTMENT  EQR  THIS  VISIT’ 

□  1.  YES 
,53‘  0  2  NO 


12 

(54  55) 


IP  YOU  HAD  AN  APPOINTMENT: 

•  ABOUT  HOW  MANY  DAYS  AGO  WAS  THE  APPOINTMENT  MADE’ 
(ENTER  0  IF  MADE  TOOAYI 
_  DAYS 

b  WHAT  WAS  THE  TIME  OF  THE  APPOINTMENT’ _  _ 


2  RANK  OP  PATIENT  OR  PATIENT'S  SPONSOR 
**  Ol  ENLISTED 
13,1  02  OFFICER 


£  PATIENT'S  OATE  OF  BIRTH 
(36441 


0»y 


j  PATIENT  S  SEX: 

n  i.  male 

1  □  2  FEMALE 

;  PATIENT'S  MARITAL  STATUS 

o  1  NOT  APPLICABLE  (PATIENT  A  CHILD) 


,4<1  O  2  SINGLE 
O  3  MARRIED 
o  4  SEPARATED/DIVORCED 
□  5  WIDOW /WIDOWER 


]  PATIENT'S  MILITARY  STATUS 

9  □  1  SPOUSE  OR  OEP'NOENT  OF  ACTIVE  MILITARY 
1471  □  2  SPOUSE  OR  OCPf  NOCNT  OF  ftCTIRCO  OR  DECEASEO  MILITARY 

□  3  ACTIVE  MIUTAA  ' 

□  4  RETIRED  MILITARY 
D  s  CIVILIAN  EMPLOYEE 


m  PATIENTS  ft  VINO  STATUS  (ACTIVE  DUTY  PERSONNEL  ONLY): 

^  □  1  ON  FLYING  STATUS 

l4#*  D  2  SUSPENDED  FROM  FLYING  STATUS 

□  3  NOT  ON  FLYING  STATUS 

Q  00  YOU  (THE  PATIENT)  LIVE  ON  THIS  BASE’ 

9  □  1  YES 

□  2  NO 


10 

ISO  S2I 


IF  "NO" 

APPROXIMATELY  HOW  MANY  MILES  AWAY’ 


13  INHAT  IS  THE  MAJOR  REASON  OR  SYMPTOM  FOR  THIS  VISIT’ 
HEAD  AMD  CHEST  AREA: 

(60  71)  p  v  COLD  OR  RUNNY  NOSE 
Q  2.  COUGH 

□  3  SORE  THROAT 

Q  4  EARACHE  OR  EAR  ORAINAGE 
Q  5  HEADACHE 

Q  6  DIZZINESSOR  LIGHTHEADEDNESS 
Q  7.  EYE  PROBLEM.  OR  CAN'T  SEE  WELL 

□  8  HEART  PROBLEM 
Q  6.  CHEST  PAIN 

STOMACH  (ABDOMINAL)  AREA: 

□  10.  INOIGESTION.  HEARTBURN.  DISCOMFORT  AFTER  EATING 
Q  It  STOMACH  (ABDOMINAL)  PAIN 

□  12  URINARY  PROBLEM  OR  INFECTION 

□  13.  PROBLEM  WITH  SEX  AREA  OF  BODY 

BONE.  MUSCLE.  SKIN 

Q  14  HURT  A  BONE.  JOINT  OR  MUSCLE 
a  IS  BACK  ACHE  OR  NECK  PAIN 

□  16  JOINT  PAIN  IOTHER  THAN  INJURY)  OR  ARTHRITIS 
O  17  SKINCUT.  SCRATCH  OR  BRUISE 

□  16.  RASH 

GENERAL: 

□  IS  FLU 

□  20  FEVER 

□  21  FATIGUE 

□  22  WEAKNESS 

□  23  NERVOUS  TENSION.  ANXIETY  DEPRESSION 

□  24  OVERWEIGHT  (OR  WEIGHT  GAIN) 

□  25  DIABETES 

□  26  HIGH  BLOOD  PRESSURE  (HYPE RTENSIONI 

□  27  THYROID  PROBLEM 

□  28  CHEMOTHERAPY 

□  26  TO  REFILL  A  PRESCRIPTION 
O  30  TO  GET  LAB  TEST 

□  31  REGULAR  (ROUTINE!  PHYSICAL  EXAM 

SOMETHING  ELSE  (WRITE  IN) - 


172  791 
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PART  II  Diagnostic  Information 


|  TO  BE  COMPLETED  BY  THE  HEALTH  CARE  PROFESSIONALS)  ONLY  ] 


19  .  STATUS  OF  FATICNT  (CHICK  All  IOXFS  THAT  AFMVI 
list  □  1  NEW  PATIENT  TO  CLINIC 
..  £ J(9)  p2  NEW  PATIENT  TO  PRACTITIONER 

b  STATUS  OF  VISIT 

<101  □  1  FIRST  VISIT  (TO  ANYONE  I  FOR  MAIN  PROBLEM  OR  PROCEDURE 
D  2  RETURN  VISIT  ITO  ANYONE!  FOR  MAIN  PROBLEM  OR  PROCEDURE 
O  3  PRESCRIPTION  REFILL  ONLY 


PO  ANY  PROPHYLACTIC  PROCEDURES  PERFORMED’  IF  YES.  CHECK 
**  APPLICABLE  BOX 

□  l  PCRIOOIC  AOULT  PHYSICAL  EXAM 
Ml  121  Q  ?  periodic  WELL  CHILD  EXAM 

D  3  FLIGHT  PHYSICAL  ICLASS  t> 

□  4  FLIGHT  PHYSICAL  ICl*  S5  2  OR  31 

□  5  SEPARATION  RETIREMENT  PHYSICAL 

□  6  OTHER  NON  FLYING  (ADMINISTRATIVE)  PHYSICAL 
Q  7  ROUTINE  EVE  EXAM 

□  S  ROUTINE  C  YN  EXAM 

□  9  PRENATAL  VISIT 

Q  10  POSTPARTUM  VISIT 

□  11  PRE  OP  VISIT 

Q  12  POST  OP  FOLLOW  UP 
Q  13  WEIGHT  CHECK  ONLY 

□  14  HAZARDOUS  NOISE  EXAM 

D  15  OTHER _ _ _ 


21  '0  NUMBER  OF  PERSONS)  TREATING  PATIENT 
**  *  (ENTER  IN  APPROPRIATE  SPACE) 


NURSE  CORPSMAN  OTHER 


22  PROBLEMS'AREAS  TREATED  ICHECK  ALL  BOXES  THAT  APPLY) 

1  COMMUNICABLE  DISEASES  (SEE  ALSO  SECTIONS  I.  10.  U.U) 

(28  421  OS  INFECTIOUS  INTESTINAL  (INCLUDES 

INFECTIOUS  OIARRHEAI 

□  23  VIRAL  SYNDROME  WITH 

GASTROENTERITIS 

□  245  VIRAL  SYNDROME  WITHOUT 

GASTROENTERITIS 

□  11  13.  15  MEASLES.  MUMPS.  CHICKEN  POX 

D  16  HEPATITIS  OR  EXPOSURE  TO  HEPATITIS 

□  IT  INFECTIOUS  MONONUCLEOSIS 

□  4  GONORRHEA  (OR  € XPOSURE  TO 

GONORRHEA) 

□  901  OTHER _ _ _ 


2  NEOPLASMS 

D  50  SB 

□  831 

□  TO  74 

□  902 


MALIGNANT  NEOPLASM 

BENIGN  SKIN  (INCLUOESMOLf  NEVUS) 

BENIGN  (OTHER  THAN  SKIN) 

OTHER  -  _  _  _ 


3  ALLERGIC.  ENOPCRINE.  METABOLIC.  NUTRITIONAL 

□  85.  95.  257  HAY  FE VERBAL L€flG)ES 

□  *e 

□  88  90 
f  )  91 

U  101 

1 1 903 


ASTHMA 

THYROID  DISEASE 
DIABETES  ME  IL  ITUS 
OBESITY 
OTHE  R - 


S  NERVOUS  SYSTEM,  MENTAL  ILLNESS.  PERSONALITY  DISORDERS 
I  j  124  129  PSYCHOSIS 

>30.  134  ANXIETY  OR  DEPRESSIVE  NEUROSIS 

LJ  633  SEXUAL  DYSFUNCTION 

600 


139 
141 
.  147 
J  159 

-  454 

;  155 

.  158  197 

156  157  159  160.  165  169 
150 

.  905 


DRUG  DEPENDENT  f  ABUSE 
ALCOHOL  ABUSE  OR  ALCOHOLISM 
TENSION  HEADACHE 
MIGRAINE  HE  AO  ACHE  (OR 
MIGRAINE  MANIFESTATIONS! 

OTHER  HEADACHE 

VASCULAR  LESIONS  (INCLUDES  STROKE 
CEREBRAL  ARTERIOSCLEROSIS) 

EPILEPSY  CONVULSIONS 
OTHER  NERVOUS  SYSTEM  DISEASES 
PROBLEM  OF  DEVELOPMENT 
RCTAROATION  OR  BEHAVIOR 


6  EYE  AND  EAR 

170  CONJUNCTIVITIS  OR  OPHTHALMIA 

.,176  REFRACTIVE  ERRORS 

178  605  STRABISMUS  TROPIA  OR  PHOBIA 

171  175.  177.  179  181  OTHER  EYE  DISEASES 

.  .  182  OTITIS  EXTERNA 

183  184  OTITIS  MEDIA  (EXCLUDES  SEROUSI 

648  SEROUS  OTiTtS  MEDIA 

.  187  WAX  IN  EAR 

.  761  185  186  1  88  1  90  OTHE R  DISEASES  OF  THE  EAR 

906  OTHER  _  _ 


7  cardiovascular 

.  .  211.  212  215 

214 

634 

1  213  215.217 
218 
:  233 

□  225 
i  224 

□  907 


ISCHEMIC  HEART  DISEASES  (INCLUOING 
ANGINA  PECTORIS  AShOI 

ARRHYTHMIAS  OR  HEART  BLOCK 

HEART  MURMUR 

OTHER  HEART  DISEASES 

HYPERTENSION  IHBPI 

SYNCOPE 

HEMORRHOIDS 

VARICOSE  VEINS 

OTHER  _ _ _ _ _ 


I  RESPIRATORY 

240 

241 

245 

242 

243 
250 
86 

747  758 

246 

248  255 

267 

263 


CORYZA  (NON  F  EBRILE  COMMON  COLO! 
FEBRILE  COLO 

INFLUENZA.  RESPIRATORY  F LU  SYNDROME 
SORE  THROAT  (PHARYNGITIS  OR  TONSILLITIS) 
ACUTE  SINUSITIS 
CHRONIC  SINUSITIS 
ASTHMA 

ACUTE  BRONCHITIS  <OR  BRONCHIOLITIS) 
PNEUMONIA  PNEUMONITIS 
CHRONIC  BRONCHITIS  EMPHYSEMA 'COPO 
COUGH  ONLY 
NOSI  BlltO 


4  BLOOD  ANO  IMMUNOLOGICAL  ORGANS 

fj  in  IRON  DEFICIENCY  (HYPOCHROM)C'  ANEMIA 

□  110  112  122  OTHER  ANEMIAS 
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9  OlGES'DVf 

□  2»3  2M  284  284 

□  272  278 

n;i< 

Os 

□  as 

□  301306 

□  200 

O  304 

□  28 6  28/ 

□  282  285  308 

O  303 

□  225 

□  283 

□  908 


ESOPHAGITIS  GASTRITIS  iNDtGE  ST  ION 
HIATAL  HERNIA 

ULCER  DISEASE  'STOMACH  OR  DUODENUM) 

OTHER  OISE  ASES  Of  ESOPHAGUS  STOMACH 
DUODENUM 

INFECTIOUS  INTESTINAL  UNCIUOES 
INFECTIOUS  DIARRHEA' 

VIRAL  SVNOROME  WITH  GASTROENTERITIS 
ABDOMINAL  pain  INOT  OTHERWISE  SPECIFIED' 
FUNCTIONAL  UPPER  Gl  DISTRESS 
FUNCTIONAL  LARGE  BOWEL  DISTRESS 
(IRRITABLE  SPASTIC  COLON' 

CHOLELITHIASIS  CHOLECYSTITIS 

OTHER  DISEASES  OF  INTESTINE  AND  PERITONEUM 

OlARRHEA 

HEMORRHOIDS 

HERNIA  INGUINAL  FEMORAL  UMBILICAL 

other  _ _ 


10  GINITQ  URINARY  SYSTEM  (SEE  ALSO  SECTION  1 1 


URINARY  TRACT  INFECTION  (CYSTITIS 
PYEUTIS'PYELONEPHRITIS' 

NONSPECIFIC  URETHRITIS  INON  GONOCOCCAL' 
GONOCOCCAL  URETHRITIS  CERVICITIS  OR 
SALPINGITIS 

OTHER  DISEASES  OF  URINARY  SYSTEM 
PROSTATITIS  OR  BENIGN  PfiOSTATIC 
HYPERTROPHY 

OTHER  DISEASES  OF  REPRODUCTIVE  SYSTEM 
VULVITIS.  VAGINITIS  AND  CERVICITIS 
(NON  VENEREAL! 

DISORDERS  OF  MENSTRUATION  DYSFUNCTIONAL 
UTERINE  BLEEDING 
MENOPAUSAL  SYMPTOMS 

FAMILY  PLANNING  CONTRACEPTION 'IN  Ft RTlLITY 
CERVICAL  EROSION 

BREAST  MASS  OR  BREAST  DISEASE  lEXCLUOING 
MALIGNANCY! 

OTHER _ _ 


Q  313  314 

□  315 

04 

□  JIT 
O  316. 318 

O  319  321.  331 
Q  335 

□  327  330  334 

D  329 
O  585.  586 
O  332 
O  322 

□  910 

12  SKIN  ANO  SUPERFICIAL  TISSUE  (SEE  ALSO  SECTION  16) 

□  21 

□  371.372.  374 

□  375 

□  369 

O  370.  373.  377 

□  378-381 

□  368 
0644 

□  3W 

□  631 
025 

□  384.  3B5 

□  387 

□  388 

□  fi  a 

13.  I0NIS.  JOINTS.  MUSCLES 

□  408 
O  405 

□  407  408 

□  420-422 

□  424  425 

□  423 

□  420 

□  «7* 

□  640 

□  642 

□  473-477 

□  488.448 

□  486 

□  807 

□  808 
□  808 
□  643 


FUNGAL  SKIN  INFECTION.  DERMATOPHYTOSlS 
CELLULITIS  HNCLUOING  LYMPHANGITIS! 
IMPETIGO 
ACNE 

OTHER  LOCAL  INFECTIONS  OF  SKIN  AND 
SUBCUTANEOUS  TISSUE 
DERMATITIS  (INCLUDING  EC2EMAI 
PITYRIASIS  ROSEA 
DRUG  RASH 

RASH  (OTHERWISE  UNSPECIFIED! 

MOLE.  NEVUS 
WARTS 

CORNS/OTHER  HYPER  TROPHIC!  A  TROPHIC 
SKIN  CONDITIONS 
DISEASES  OF  NAIL  AND  NAIL  BED 
(EXCLUDING  FUNGUS) 

DISEASES  OF  SWEAT  ANO  SEBACEOUS  GLANDS 
(INCLUDING  SEBACEOUS  CYST) 

OTHER  - _ 


RHEUMATOID  ARTHRITIS 
OSTEOARTHRITIS 

OTHER  ARTHRITIS/RHEUMATISM.  INCLUDING 
POST  TRAUMA 

BURSITIS.  TENOSYNOVITIS.  SYNOVITIS 
BACKACHE  ALONE.  BACK  PAIN  ALONE 
BACKACHE  WITH  SCIATICA 
PAIN  IN  JOINT  (ARTHRALGIA) 

DISLOCATION.  UPPER  EXTREMITY 
DISLOCATION.  LOWER  EXTREMITY 
TRAUMA  TO  HEAD 
FRACTURE  OF  UPPER  LIMB 
FRACTURE  OF  LOWER  LIMB 
OTHER  FRACTURE 
SPRAIN/STRAIN  UPPER  LIMB 
SPRAIN/STRAIN  LOWER  LIMB 
SPRAIN /SYR  AIN  NECK/BACK 
MUSCLE  PAIN.  MUSCLE  CRAMPS 


16  ACCIDENTS.  POISONINGS  ANO  VIOLENCE  ISEE  ALSO  SECTION  13) 

465  484  FORI  IGN  BODY  'DEFINITE  OR  PQSSifll  £  ■ 

483  LACERATIONS  CONTUSIONS  ABRASIONS 

SUPERFICIAL  INJURIES 

394  INSECT  Bites 

61 1  ANIMAL  BITES 

405  487  BURNS 

488  491  POISONING  OVEROOSE 

9W  OTHER _ 


W  SIGNS  SYMPTOMS  ANO  III  OEFINEO  CONDITIONS 

45S  MALINGERING 

458  FEVER  OF  UNKNOWN  ORIGIN 

646  PAIN 

731  ? 71  CHEST  PAIN 'UNKNOWN  ETIOLOGY 

647  NO  PATHOLOGY  AT  THIS  TIME  'WYE  LL 
PATIENT) 

816  NO  DEFINITE  DIAGNOSIS  AT  THIS  TIME 


19  OTHER 

919 


OTMf 


R 


23  StmOUSNESS  of  MAIN  CONDITION  TREATED  (CHECK  ONEl 

"  □  »  VERY  SERIOUS 
1431  [  }  2  SER'OUS 

□  3  SLIGHTLY  SERIOUS 

□  4  NOT  SERIOUS 


2 A  disposition  of  visit 

□  i  NO  FORMAL  FOLLOW  UP  PLANNED 
(44  491  q  2  RETURN  PRN  (POSSIBLE  FOLLOW  UP* 

Q  3  DEFINITE  TELEPHONE  FOLLOW  UP 

□  4  DEFINITE  RETURN  APPOINTMENT 

□  5  REFER  TO  OTHER  PROFESSIONAL  OR  CLINIC  ON  THIS  VISIT 

ISPECIFY  BY  10  NUMBER  OR  NAME) 


Q  6  REFER  TO  OTHER  PROFESSIONAL  OR  CLINIC  FOR  FUTURE 
APPOINTMENT  (SPECIFY  BY  ID  NUMBER  OR  NAME) 


Q  7  REFER  TOCHAMPUS 

□  8  ADMIT  TO  QUARTERS 

□  9  ADMIT  TO  HOSPITAL 

□  10  RETURN  TO  REFERRING  CLINIC  ISPECIFY  BY  NAME) 


□  it  other 


150  52* 


25  »  DIO  YOU  SPEAK  OR  CONSULT  WITH  ANOTHER  HEALTH  CARE 
PROFESSIONAL  CONCERNING  THIS  PATIENT  » 

(53!  Ql  YES  - AND  OTHER  PROFESSIONAL  SAW  PATIENT  ON  THIS  VISIT 
O?  YES-SPOKE.  BUT  OTHER  PROFESSIONAL  DIO  NOT  SEE  PAT'ENT 
ON  THIS  VISIT 

□  3  YES-BUT  ONLY  TO  HAVE  A  PRESCRIPTION  OR  ORDER 

COUNTERSIGNED 

□  •  NO 

154  S6>  b  IF  YES.  ENTER  NAME  OR  10  NUMBER  OF  OTHER  PROFESSIONAL 
CONSULTED  _ _ 
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PART  III  Treatment  Information 


IN  THE  QUESTIONS  WHICH  FOLLOW.  EACH 
HEALTH  PROFESSIONAL  SHOULD  CHECK  OFF 
APPROPRIATE  BOXES  IN  HIS  OR  HER  COLUMN 


26 

TESTS  (CHECK  ALL  APPLICABLE  BOXES) 

i  DIAGNOSTIC  TESTS  (ORDERED 

OR  PERFORMED)  DR 

PA 

7 

NURSE 

3 

CORPS 

MAN 

OTHER 

(57) 

audiogram 

D 

□ 

G 

□ 

n 

<581 

CULTURE-  GENITAL  TRACT  OR 

WET  PREP  SMEAR  OR  GRAM  STAIN 

□ 

D 

G 

□ 

G 

1591 

CULTURE  THROAT 

□ 

G 

G 

G 

□ 

1601 

CULTURE -URINE 

□ 

□ 

D 

a 

a 

1611 

CULTURE  OTHER  SOURCES 

ISPUTUM  STOOL  ETC  » 

□ 

a 

G 

G 

a 

162) 

EXERCISE  (STRESS)  TEST 

□ 

G 

n 

G 

□ 

163) 

PAP  SMEAR 

n 

G 

G 

□ 

a 

(64) 

PULMONARY  FUNCTION  TESTS 

□ 

a 

a 

□ 

a 

165) 

SKIN  TESTS  (ALLERGY  TB.  ETC  ) 

n 

a 

a 

G 

□ 

1671 

URINALYSIS 

□ 

□ 

a 

□ 

D 

168) 

OTHER 

a 

a 

o 

O 

n 

[card  3[ 

18) 

b  LAB  TESTS  ORDERED 

HEMOGLOBIN/HEMATOCRIT  (ONLY) 

a 

o 

o 

□ 

□ 

(91 

WBCAVHITE  BLOOD  OOUN T  (WITH 

OR  WITHOUT  DlFF  1 

□ 

a 

□ 

□ 

n 

(10) 

COMPLETE  BLOOD  COUNT/CBC 

□ 

a 

□ 

G 

a 

(111 

SYPHILIS  SEROLOGY  VDRl 

□ 

□ 

n 

U 

□ 

(12) 

MONOSPOT  OR  HETEROPHILE 

□ 

G 

□ 

□ 

n 

(131 

URINALYSIS 

□ 

a 

G 

G 

n 

(14) 

BLOOD  SUGAR  (SINGLE  OR 
TOLERANCE  TEST) 

□ 

□ 

G 

□ 

a 

115) 

CHOLESTEROL/TRIGLYCERIDES 

(LIPIDS) 

□ 

G 

□ 

a 

□ 

(16) 

BUN/CREATININE 

□ 

a 

a 

a 

□ 

(171 

ELECTROLYTES  (ONE  OR  MOREl 

□ 

a 

G 

a 

□ 

<181 

LIVER  FUNCTION  TESTIS) 

□ 

Q 

n 

□ 

n 

(19) 

THYROIO  IT3.  T4.  TSH.  ETC  1 

□ 

□ 

n 

□ 

a 

120) 

EKG 

□ 

n 

a 

□ 

D 

(21) 

X  RAY  CHEST 

a 

a 

n 

n 

n 

<221 

X  RAY  OTHER  THAN  CHEST 

a 

D 

a 

n 

a 

173) 

OTHfll 

D 

n 

□ 

□ 

n 

124) 

c  SELECTED  EXAMINATION 
PROCEDURES  PERFORMED 

BLOOO  PRESSURE  MEASUREO 

n 

n 

G 

n 

G 

1751 

LUNGS  AUSCULATEO 

□ 

a 

rj 

Cl 

n 

176) 

HEART  AUSCULATEO 

n 

n 

n 

n 

i] 

(271 

ABOOMEN  PALPATED 

n 

u 

n 

ti 

n 

(28( 

PELVIC  EXAMINATION  DONE 

n 

n 

n 

n 

n 

i  PHYStCAl  MEASURES 

1291  SUTURE  PLACEMENT  OR  REMOVAL 
<301  APPLY.  CHECK.  CHANGE  DRESSING 
<311  REMOVE  FOREIGN  BODY 
(32)  INCISION.  EXCISION  OR  BIOPSY 
<331  ORTHOPEDIC  PROCEDURE 
(341  ORDEH/GIVE  PHYSICAL  THERAPY 
(351  IMMUNIZATION  OR  VACCINATION 
IORDERED  OR  DONE! 

(361  INJECTION  OTHER  THAN  IMMUN  / 
VACC  IORDERED  OR  DONE) 

<371  CONTRACEPTIVE  PROCEDURE 
IINCLUDES  IUD) 

1381  OTHER  _  _ _ 

b  SELECTED  MEDICATIONS 
(WHETHER  STARTED  OR 

CONTINUED) _ 

1391  ANALGESIC- NARCOTIC  (OR 
COMBINATION  NON  NARCOTIC 
WITH  NARCOTIC! 

140}  ANALGESIC-NON  NARCOTIC 
(41)  ANTACIO,  ANTI  EMETIC. 

ANTIOIARRHEAL 
(421  ANTI  ANEMIA  IHEMATINICI 
ANTIBIOTICS 

143)  ANY  PENICILLIN  IINCLUOING 
AMPICILUNI  OR  ERYTHROMYCIN 

144)  ANY  TETRACYCLINE 
1451  ANY  SULFA  DRUG 

146)  ANY  OTHER  ANTIBIOTIC 
1471  ANTIHlSTAMINlC/DECONGESTANT  < 
ANTITUSSIVE/ANTI  COLO 
148)  ANTIHYPERTENSIVE  AND/OR 
OIURETIC 

(491  ANTI  INFLAMMATORY 
(50-  BRONCHIAL  DILATOR 
1571  CARDIOVASCULAR  IDIG  .  ANTI 
ANGINAL.  ANTI  ARRHYTHMIC) 

1521  CONTRACEPTIVE  (ORAL)  OR 
ESTROGEN  PREPARATION 
(531  INSULIN  OR  ORAL  MYPOGLY 
CEMIC  AGENT 
1541  SEOATIVE/SLEEPMEO 
(55)  STEROID  ORAL  OR  INJECTED 
(5C>  STEROID -TOPICAL  OR  INHALED 
IS7I  THYROID  RELATED  MEO 
158)  TRANQUIH7ER  OR 
ANTI  OCPRESSANT 
(591  VITAMINS  ANO  MINERALS 
INC  IRON 
IB0>  OTHER 

c  COUNSEL  me  ARP  OTHER 
(Si)  COUNSEL  ABOUT  DISEASE.  MIDI 
CATION  OR  OTHER  TREATMENT 
f*2>  ADVICE  ABOUT  CONTRACEPTION 
1831  PSYCHOLOGICAL  COUNSELING 
1841  CHAPERONING 
(851  ADMINISTRATIVE  ACTION 
I88>  nrniw 


CORPS 

OR 

PA 

NURSE 

MAN 

OTHER 

1 

7 

3 

4 

5 

a 

□ 

a 

a 

a 

a 

□ 

□ 

a 

a 

□ 

□ 

□ 

□ 

□ 

□ 

G 

a 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

o 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

a 

□ 

□ 

a 

□ 

□ 

□ 

a 

a 

□ 

a 

□ 

□ 

a 

□ 

a 

a 

□ 

□ 

a 

n 

□ 

□ 

□ 

D 

□ 

r\ 

Q 

□ 

□ 

D 

5 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

D 

a 

□ 

□ 

a 

□ 

□ 

D 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

D 

n 

□ 

□ 

□ 

□ 

□ 

□ 

D 

□ 

D 

□ 

D 

n 

a 

□ 

□ 

D 

□ 

a 

a 

□ 

□ 

D 

□ 

D 

a 

D 

□ 

a 

D 

□ 

□ 

G 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

a 

a 

a 

a 

G 

D 

□ 

a 

a 

□ 

n 

a 

□ 

□ 

□ 

o 

□ 

a 

a 

□ 

c 

□ 

Q 

a 

□ 

c 

□ 

□ 

D 

□ 

□ 

□ 

□ 

a 

a 

□ 

□ 

D 

□ 

□ 

a 

□ 

□ 

□ 

PART  IV  Time  Information 


PATIENT  SHOUIO  ANSWER  THE  FOLLOWING 
QUESTIONS  AT  THE  END  OF  THE  VISIT 


•  MOW  MUCH  TIME  WAS  SPENT  WITH  THE  NURSE.  OH  NURSE  PRACTITIONER. 

IF  ANY’ 


1 89 '  LESS  Than  1  5  5  10  10  20  20  30  MORE  THAN 

NONE  A  MIN  MIN  MIN  MIN  MIN  30  MIN 
0  1  2  3  4  5  8 

; ;  u  □ 


]  A  DURING  YOUR  VISIT  APPROXIMATELY  HOW  MUCH  TIME  WAS  SPENT  WITH 
THE  DOCTOR  IF  ANY’ 

1671  LESS  Than  15  5  10  1020  20  30  MORE  than 

NONE  AMIN  ttIN  MIN  MIN  MIN  30  MIN 
0  t  2  3  4  5  6 


NOW  MUCH  TIME  WAS  SPENT  WITH  THE  CORPSMAN.  IF  ANY’ 

<70>  less  than  is  5  to  1020  ?o  30  more  than 

NONE  A  MIN  MIN  MIN  MIN  MIN  30  MIN 
0  1  2  3  4  5  6 

I  J  L-]  U 


15 


HOW  MUCH  TIME  WAS  SPENT  WITH  THE  PHYSICIAN  ASSISTANT 
(I0CNTIFIE0  BY  THE  HUE  EMBLEM  PA  ON  HIS  JACKET)  IF  ANY’ 


IS  HOW  MUCH  TIME  WAS  SPENT  WITH  OTHER  HEALTH  CARE  PROFESSIONALS, 
■•if  ANY’ 


LESS  THAN  1  5  5  10  10  20  20  30  MORE  THAN 

AMIN  MIN  MIN  MIN  MIN  30  MIN 
1  7  3  4  5  6 


LESS  THAN  1  5  5  10  10  20  20  30  MORE  THAN 
AM(N  MIN  MIN  MIN  MIN  30  MIN 

1  2  3  4  5  • 


NONE 


NONE 
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Table  A.l 

Frequencies  of  Conditions  Marked  by  Selected  Practitioners  at 
Dyess,  Fairchild,  Chanute,  and  Nellis  Air  Force  Bases: 

1977  Patient  Contact  Record 


Most  Serious  Diagnosis: 

Code  Number  and  Description 

MD 

Only 

PA 

Only 

PCNP 

Only 

2 .  Jonmio: 

rlcable  i>isease8 

*5 

Infectious  intestinal 

(incl.  infectious  diarrhea) 

4 

5 

1 

*23 

Viral  syndrome  w/gastroenteritis 

104 

160 

51 

*245 

Viral  syndrome  w/o  gastroenteritis 

148 

191 

52 

11-13, 

15  Measles ,  mumps ,  chicken  pox 

8 

11 

7 

16 

Hepatitis  or  exposure  to  hepatitis 

11 

8 

4 

17 

Infectious  mononucleosis 

1 

15 

2 

*4 

Gonorrhea  (or  exposure  to  gonorrhea) 

17 

28 

10 

901 

Other 

21 

18 

1 

Total 

314 

436 

128 

%  of  practitioner  total 

6.0% 

5.3% 

6.2% 

£ .  iit iOfr  uUSl'ld 

50-68 

Malignant  neoplasm 

30 

4 

2 

*631 

Benign  skin  (incl.  mole/nevus) 

8 

24 

11 

70-74 

Benign  (other  than  skin) 

3 

4 

5 

902 

Other 

7 

5 

3 

Total 

48 

37 

21 

%  of  practitioner  total 

0.9% 

0.5% 

1.0% 

3.  niters 

la  Endocrine,  Metabolic,  Nutritional 

85,  95“ 

257  Hay  fever/allergies 

100 

248 

61 

*86 

Asthma 

59 

61 

14 

88-90 

Thyroid  disease 

57 

93 

15 

91 

Diabetes  mellitus 

90 

78 

10 

101 

Obesity 

84 

65 

9 

903 

Other 

41 

39 

13 

Total 

431 

584 

122 

%  of  practitioner  total 

8.2% 

7.1% 

5.9% 

4.  Blood 

and  Irmunologioal  Organs 

111 

Iron-deficiency  (hypochromic)  anemia 

14 

12 

2 

110,  112,  122  Other  anemias 

4 

6 

1 

904 

Other 

14 

8 

2 

Total 

32 

26 

5 

%  of  practitioner  total 

0.6% 

0.3% 

0.2% 
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Table  A.1 — continued 


Most  Serious  Diagnosis: 

MD 

PA 

PCNP 

Code 

Number  and  Description 

Only 

only 

Only 

o.  .Veinv-utf  oyster 

,  Mental  1  linear ,  i-cvaonxlitj  -  Ls-.rUefiS 

124-129 

Psychosis 

5 

2 

0 

130,  134 

Anxiety  or  depressive  neurosis 

50 

38 

10 

633 

Sexual  dysfunction 

1 

1 

0 

uOO 

Adult  situation  disturbance 

(e.g.,  marital) 

1 

22 

3 

139 

Drug  dependence /abuse 

6 

2 

0 

141 

Alcohol  abuse  or  alcoholism 

12 

3 

0 

147 

Tension  headache 

19 

39 

6 

159 

Migraine  headache  (or  other 

migraine  manifestations) 

25 

29 

8 

454 

Other  headache 

18 

44 

11 

155 

Vascular  lesions  (incl.  stroke. 

cerebral  arteriosclerosis) 

10 

3 

0 

158,  197 
156-157, 

Epilepsy  convulsions 

30 

17 

2 

159-160, 

165-169 

Other  nervous  system  diseases 

29 

24 

15 

150 

Problem  of  development,  retardation, 

or  behavior 

1 

4 

1 

905 

Other 

17 

24 

3 

Total 

224 

252 

59 

/.  of  practitioner  total 

4.3/. 

3.1% 

2.9% 

C,  c'jc  and  zdr 

170 

Conjunctivitis  or  ophthalmia 

50 

109 

41 

176 

Refractive  errors 

4 

3 

1 

178,  605 
171-175,  177, 

Strabismus,  Cropia,  or  phoria 

1 

3 

2 

179-181 

Other  eye  diseases 

52 

53 

15 

182 

Otitis  externa 

35 

68 

21 

183,  184 

Otitis  media  <excl.  serous) 

104 

197 

50 

648 

Serous  otitis  media 

25 

247 

54 

187 

161,  185-186, 

Wax  in  ear 

15 

34 

17 

188-190 

Other  diseases  of  the  ear 

24 

60 

13 

906 

Other 

29 

13 

7 

Total 

339 

78  7 

221 

%  of  practitioner  total 

6.5% 

9.6% 

10.7% 

211,  212-215 

Ischemic  heart  diseases  (incl.  angina 

pectoris,  ASHD) 

116 

13 

8 

214 

Arrhythmias  or  heart  block 

62 

16 

6 

6  34 

heart  murmur 

43 

12 

4 

213,  215,  217 

Other  heart  disease 

15 

19 

1 

218 

Hypertension  (HBP) 

220 

417 

101 

233 

Syncope 

5 

6 

2 

*223 

Hemorrhoids 

13 

45 

10 

224 

Varicose  veins 

4 

8 

3 

907 

Other 

24 

25 

6 

Total 

502 

561 

141 

4  of  practitioner  total 

9.6% 

6.8% 

6.8% 

44 


Table  A.l — continued 


Most  Serious  Diagnosis: 

MD 

PA 

PCNP 

Code 

Number  and  Description 

Only 

Only 

Only 

8.  Respiratory 

240 

Coryza  (nonfebrile  common  cold) 

225 

587 

176 

241 

Febrile  cold 

87 

86 

65 

245 

Influenza,  respiratory  flu  syndrome 

148 

191 

52 

242 

Sore  throat  (tonsillitis  or 

pharyngitis) 

241 

429 

113 

243 

Acute  sinusitis 

46 

101 

22 

250 

Chronic  sinusitis 

31 

45 

14 

86 

Asthma 

59 

61 

14 

247,  258 

Acute  bronchitis  (or  bronchiolitis) 

45 

189 

29 

246 

Pneumonia,  pneumonitis 

14 

58 

23 

248,  255 

Chronic  bronchitis/ emphysema /COPD 

19 

40 

10 

267 

Cough  only 

18 

39 

12 

263 

Nose  bleed 

7 

8 

3 

908 

Other 

37 

46 

8 

Total 

977 

1880 

541 

%  of  practitioner  total 

18.7% 

22.9% 

26.2% 

9.  Digestive 

273,  274, 

Esophagitis,  gastritis, 

284,  294 

indigestion,  hiatal  hernia 

40 

96 

21 

277-279 

Ulcer  disease  (stomach  or  duodenum) 

18 

36 

4 

281 

Other  diseases  of  esophagus. 

stomach,  duodenum 

3 

15 

3 

5 

Infectious  intestinal  (incl. 

infectious  diarrnea) 

4 

5 

1 

23 

Viral  syndrome  w/gastroenteritis 

104 

160 

51 

301.  306 

Abdominal  pain  (not  otherwise 

specified) 

36 

82 

29 

280 

Functional  upper  GI  distress 

14 

18 

5 

304 

Functional  large  bowel  distress 

(irritable/spastic  colon) 

7 

27 

5 

286,  287 

Cholelithiasis,  cholecystitis 

10 

12 

4 

282,  285,  309 

Other  diseases  of  intestine  and 

peritoneum 

22 

13 

1 

303 

Diarrhea 

18 

22 

7 

225 

Hemorrhoids 

13 

45 

10 

283 

Hernia — inguinal,  femoral,  umbilical 

2 

15 

3 

909 

Other 

53 

52 

19 

Total 

344 

598 

163 

%  of  practitioner  total 

6.6% 

7.3% 

7.9% 

45 


Table  A.l — continued 


Most  Serious  Diagnosis: 

Code  Number  and  Description 

MD 

Only 

PA 

Only 

PCMP 

Only 

10.  Geni to -Urinary  System 

313,  314  Urinary  tract  infection  (cystitis/ 

pyelitis/pyelonephritis) 

72 

193 

54 

315 

Nonspecific  urethritis 
(nongonococcal) 

19 

55 

14 

4 

Gonococcal  urethritis,  cervicitis, 
or  salpingitis 

17 

28 

10 

317 

Other  diseases  of  urinary  system 

18 

29 

9 

316,  318 

Prostatitis  or  benign  prostatic 
hypertrophy 

7 

16 

2 

319-321,  331 

Other  diseases  of  reproductive  system 

14 

33 

7 

335 

Vulvitis,  vaginitis,  and  cervicitis 
(non venereal) 

8 

19 

6 

327-330,  334 

Disorders  of  menstruation, 

dysfunctional  uterine  bleeding 

17 

12 

4 

329 

Menopausal  symptoms 

1 

3 

0 

585,  586 

Family  planning/contraception/ 
infertility 

2 

12 

1 

332 

Cervical  erosion 

0 

0 

0 

322 

Breast  mass  or  breast  disease 
(excl.  malignancy) 

9 

24 

17 

910 

Other 

27 

34 

10 

Total 

211 

458 

134 

%  of  practitioner  total 

4.02 

5.62 

6.52 

22.  Skin  and  Superficial  Tissue 

21  Fungal  akin  Infection, 

dermatophytosis 

44 

83 

22 

371,  372,  374 

Cellulitis  (incl.  lymphangitis) 

10 

43 

3 

375 

Impetigo 

3 

19 

6 

369 

Acne 

41 

137 

19 

370,  373,  377 

Other  local  infections  of  skin  and 
subcutaneous  tissue 

30 

92 

36 

378-381 

Dermatitis  (incl.  eczema) 

55 

148 

33 

368 

Pityriasis  rosea 

2 

6 

1 

644 

Drug  rash 

3 

4 

2 

398 

Rash  (otherwise  unspecified) 

42 

125 

20 

631 

Mole,  nevus 

8 

24 

11 

25 

Warts 

21 

42 

4 

384,  385 

Corns/other  hypertrophic /atrophic 
skin  conditions 

8 

19 

5 

387 

Diseases  of  nail  and  nailbed 
(excl.  fungus) 

7 

15 

6 

389 

Diseases  of  sweat  and  sebaceous 
glands  (incl.  sebaceous  cyst) 

18 

53 

4 

912 

Other 

37 

72 

ii 

Total 

329 

882 

183 

%  of  practitioner  total 

6.32 

10.72 

8.92 

Table  A.l — continued 


Host  Serious  Diagnosis: 

Code  tf umber  and  Description 

MD 

Only 

PA 

Only 

PCNP 

Only 

13.  Bones , 

Joints, 

Muscles 

406 

Rheumatoid  arthritis 

38 

25 

1 

405 

Osteoarthritis 

12 

74 

3 

407-409 

Other  arthritis/rheumatism. 

incl.  post-trauma 

27 

59 

14 

420-422 

Bursitis,  tenosynovitis,  synoviti6 

34 

106 

22 

424-425 

Backache  alone,  back  pain  alone 

64 

189 

57 

423 

Backache  with  sciatica 

22 

33 

12 

428 

Pain  in  joint  (arthralgia) 

29 

97 

25 

479 

Dislocation,  upper  extremity 

5 

4 

0 

640 

Dislocation,  lower  extremity 

0 

1 

0 

642 

Trauma  to  head 

26 

17 

2 

473-477 

Fracture  of  upper  limb 

26 

25 

4 

468,  469 

Fracture  of  lower  limb 

15 

9 

1 

495 

Other  fracture 

11 

10 

1 

607 

Sprain/s train  upper  limb 

59 

64 

16 

608 

Sprain/strain  lower  limb 

81 

156 

23 

609 

Sprain/strain  neck/back 

58 

71 

22 

643 

Muscle  pain,  muscle  cramps 

59 

132 

37 

913 

Other 

86 

133 

40 

Total 

652 

1205 

280 

%  of  practitioner  total 

12.5% 

14.7% 

13.6% 

16.  Accidents,  Poisonings,  and  Violence 

465,  484 

Foreign  body  (definite  or  possible) 

9 

9 

0 

483 

Lacerations,  contusions,  abrasions. 

superficial  injuries 

186 

167 

22 

394 

Insect  bites 

1 

3 

1 

611 

Animal  bites 

17 

6 

1 

485-487 

Bums 

14 

16 

4 

488-491 

Poisoning,  overdose 

10 

0 

0 

917 

Other 

18 

19 

3 

Total 

255 

220 

31 

%  of  practitioner  total 

4.9% 

2.7% 

1.5% 

17.  Signs, 

Symptoms,  Ill-defined  Conditions 

455 

Malingering 

0 

0 

0 

458 

Fever  of  unknown  origin 

7 

6 

1 

646 

Fain 

3 

13 

5 

231,  271 

Chest  pain  (unknown  etiology) 

28 

19 

5 

647 

Mo  pathology  at  this  time 

(well  patient) 

15 

89 

24 

916 

No  definite  diagnosis  at  this  time 

58 

94 

50 

Total 

111 

221 

85 

X  of  practitioner  total 

2.1% 

2.7% 

4.1% 

19.  Other 

919 

Other 

38 

23 

15 

X  of  practitioner  total 

0.7%  0.3% 

0.7% 

Table  A.l — continued 


Most  Serious  Diagnosis:  MD  PA  PCNP 

Code  Humber  and  Description  Only  Only  Only 

Ho  diagnosis  listed  778  564  81 

%  of  practitioner  total  14. 9%  6.9%  3.9% 

Total,  all  diagnoses  .  5585  8734  2210 

Minus  double-counted  diagnoses  . . .  -353  -514  -149 

Practitioner  total  .  523 2  8220  2061 


These  diagnoses  are  repeated  under  later  categories; 
double-counted  in  this  table  and  cause  the  practitioner 
100  percent. 


therefore  they  are 
totals  to  exceed 


Appendix  B 

STATISTICAL  TECHNIQUES 


r. 


r 


TESTS  FOR  INDIVIDUAL  CRITERIA 

We  used  two  alternative  approaches  in  testing  whether  practitioner  performance  on  each 
individual  criterion  differed  significantly.  The  first  technique  compares  the  compliance  rates 
for  all  visits  (under  the  criterion)  to  one  practitioner  type  with  the  compliance  rates  for  another 
practitioner  type.  The  second  technique  uses  logit  regressions  to  estimate  differences  in  com¬ 
pliance  rates  due  to  practitioner  type  and  also  due  to  the  practitioners’  base.  Unlike  the 
compliance  rate  techniques,  the  regression  technique  allows  us  to  test  for  differences  among 
practitioners  while  controlling  for  base  differences. 

Compliance  Rates 

For  each  criterion  we  investigated,  we  calculated  rates  for  each  of  the  three  groups  of 
providers:  physicians,  physician’s  assistants  (PAs),  and  primary  care  nurse  practitioners 
(PCNPs).  These  rates  reflect  the  number  of  times  providers  in  a  group  performed  the  action 
specified  by  each  quality-of-care  criterion,  divided  by  the  number  of  visits  to  the  group’s 
providers  that  apply  to  the  criterion.  Here  we  describe  how  we  determined  whether  the 
observed  differences  in  performance  rates  are  statistically  significant,  i.e.,  are  likely  to  reflect 
real  differences  among  practitioner  types. 

A  group’s  performance  depends  on  the  set  of  decisions  each  group  member  makes.  Each 
practitioner  will  see  a  number  of  patients  for  a  particular  problem.  The  practitioner’s  pattern 
of  treatment  for  this  problem  determines  his  or  her  compliance  level  (py)  for  the  criterion 
applying  to  the  problem;  thus,  py  is  the  underlying  per-visit  probability  that  the  ith  individual 
of  the  jth  practitioner  group  will  comply.  Expected  compliance  by  a  group,  pj,  depends  on  the 
individual  compliance  of  its  members  and  on  how  the  caseload  is  distributed  among  the  group 
members: 


‘“j 

Pj  =  2  WyP; 
i=  1 

For  visits  described  by  a  particular  criterion,  Wy  represents  the  probability  the  visit  is  handled 
by  the  ith  individual  in  the  jth  practitioner  group;  mj  is  the  number  of  individuals  in  the  jth 
group. 

Our  test  for  statistical  significance  directly  compares  group  performances,  the  Pj’s,  two  at 
a  time.  To  use  the  test,  we  must  assume  that  Pj  represents  the  probability  of  compliance  for 
each  and  every  member  of  group  j,  ignoring  the  fact  that  within  group  j,  members  vary  in  their 
compliance  with  a  particular  criterion.  With  this  assumption,  we  can  define  compliance  as  the 
result  of  Hj  simple  Bernoulli  trials  with  probability  of  success  on  each  trial  pj(  n^  being  the 
number  of  visits  to  practitioners  from  the  jth  group  where  the  criterion  applies.  The  number 
of  successes,  or  compliances,  in  nj  visits  is  a  binomial  random  variable  with  parameters  Pj  and 
nj 

We  cannot  actually  observe  the  underlying  compliance  probabilities  for  individuals  (Py)  or 
groups  (pp.  Instead  we  estimate  Pj  by  calculating  the  actual  compliance  rates  (pp  achieved 
during  our  data  collection  period. 
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We  can  test  whether  the  observed  difference  between  two  groups’  performances  is  statis¬ 
tically  large  by  standard  tests  of  the  difference  in  two  proportions.  The  test  statistic  used  is: 


where 

P 


djk 


Pj  '  Pk 


ni  +  nk  - 
- - *  (P)  (1-P) 


ntn 


jk 


test  statistic  for  differences  between  groups  j  and  k; 

njPj  +  nkPk 


combined  compliance  rate  of  both  groups  =  - 


nj  +  nk 


With  large  enough  numbers  of  visits,  djk  is  distributed  approximately  as  a  standard  normal 
variable. 

Selection  of  the  critical  value  (K)  for  the  test  depends  on  the  fact  that  two  comparisons 
against  physicians’  performances  are  made,  one  for  PAs  and  one  for  PCNPs.  To  limit  to  the 
5  percent  level  the  chances  of  erroneously  concluding  that  both  PAs  and  PCNPs  differ,  we 
selected  a  critical  value  of  2.24  for  each  of  the  comparisons,  MD  with  PA  and  MD  with  PCNP. 
This  gives  a  5  percent  limit  to  the  chances  of  erring  in  the  pair  of  conclusions.1 


Power  of  the  Test 

Many  of  the  criteria  that  are  reported  in  Appendix  C  have  so  few  cases  that  the  signifi¬ 
cance  tests  may  have  little  meaning  when  they  fail  to  reject  the  hypothesis  of  equality.  With 
so  few  observations,  the  chances  of  accepting  the  hypothesis  of  equality  between  practitioner 
types  can  be  very  large  even  if  true  differences  exist  between  the  practitioners’  compliance 
rates. 

Criteria  that  were  used  in  Table  4  for  the  summary  comparisons  were  chosen  based,  in 
part,  on  the  power  of  each  test.  We  included  in  the  summary  only  those  criteria  with  sufficient 
observations  to  result  in  a  75  percent  chance  of  getting  a  significant  difference  when  the  true 
difference  between  compliance  rates  (pj  —  pk)  was  0.25  or  greater.  Thus  we  protect  ourselves 
from  the  chances  of  erroneously  concluding  that  the  compliance  rates  are  equal  when  they 
actually  differ  by  a  considerable  amount. 

To  calculate  the  power  of  the  test,  we  begin  by  specifying  the  alternate  hypothesis,  H„:  p, 
£  pk.  The  power  of  the  test  (<j>)  is  the  probability  of  rejecting  the  null  hypothesis  (H0:  p,  =  pk), 
given  the  true  values  of  p;  and  pk  and  the  sizes  of  the  two  samples  nj  and  nk.  Thus,  4>  is  the 
probability  the  test  statistic,  previously  calculated,  exceeds  the  selected  critical  value  K, 
2.24  in  our  case. 

We  want  to  calculate 


<t>  =  Pr(djk  >  K). 

Multiplying  both  sides  of  this  expression  by  the  denominator  of  djk  and  substituting  p  for  p, 
we  get  the  following: 


'White  this  may  aeem  a  rather  conaervative  choice  for  the  critical  value,  using  only  criteria  with  adequately  high 
estimated  power  protects  against  erroneously  concluding  that  compliance  rates  are  equal  when  they  in  fact  differ. 


so 


<*»  =  Pr 


Pj-  Pk>K/nj+  nk  _ 


n;n 


jk 


P(l-P) 


Since  (Pj— Pk)  is  approximately  normally  distributed  with  mean  (pj — pK)  and  variance  equal 
to 

nk.Pj(l  —  Pj)  +  njPkd-Pk) 

njnit 

we  can  standardize  (p;— pk)  by  subtracting  the  mean  and  dividing  by  the  square  root  of  the 
variance,  giving  z  with  a  standard  normal  distribution: 


=  Pr 


Z  > 


K 


lnj  +  n 


n;n 


j  k 


P  (1-P)  -  (Pj'Pk) 


nkPj  (I-  Pj)  *  njpk  (1-  Pk) 
njnk 


Thus,  the  power  of  the  test  will  be  0.75  or  greater  if  the  following  inequality  holds: 


fnj  +  nk 


K 


mn 


j  k 


-P  (1-P)  -  (Pj-Pk) 


nkPj(l-Pj)  +  njPk(l-pk) 


njnk 


<  -0.68  , 


(—0.6 8)  being  the  25th  percentile  of  the  standard  normal  distribution. 

Since  this  expression  depends  on  the  assumed  values  for  pj  and  pk,  we  must  make  some 
restrictive  assumptions  about  these  two  parameters.  First,  we  want  the  power  to  be  applicable 
to  the  situation  where  the  true  values  of  the  compliance  rates  differ  by  at  least  0.25.  Second, 
we  will  assume  that  p  =  p.  These  conditions  define  a  system  of  two  equations  with  only  two 
unknowns,  pj  and  pk: 

(1)  Pj  -  pk  =  0.25 

(2)  niPi  +  n»P>  =  Pjft  +  nnPk> 

njnk  n^k 


Solving  these  two  equations  gives  the  Pj  and  pk  values  to  be  used  in  the  power  cal- 
cluations.  If  p  is  very  small  or  very  large,  then  no  pair  of  p]  and  pk  may  yield  the  cor¬ 
rect  p  for  the  given  values  of  nj  and  nk.  We  then  must  choose  a  pair  of  p3  and  pk  that 
yield  a  larger  p  to  estimate  the  power. 
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The  power  calculations  were  done  on  the  42  criteria  that  were  selected  as  representative 
of  the  entire  set  of  criteria.  Tables  C.l  to  C.5  show  which  criteria  were  included  in  the  summary 
comparisons  because  they  had  enough  cases  to  meet  the  power  requirement.  To  give  an 
indication  of  the  range  of  required  sample  sizes,  we  include  Table  B.l,  which  shows  the  sample 
size  required  for  power  level  of  0.75  with  the  given  values  of  pj  and  pk,  and  assuming  that 
n,  -  nk  or  that  nj  =  2nk. 


Table  B.l 

Sample  Sizes  Required  for  Power  of  0.75,  for  Different 
Values  of  Pj  and  Pk,  Assuming  that  Pj  -  Pk  =  0.25 
and  Either  Nj  =  Nk  or  Nj  =  2Nk 


If  »j  *  \ 

If  N 

fN 

II 

p. 

j 

Pk 

« . 

J 

\ 

.60 

1  .35 

68 

10] 

50 

.50 

.25 

63 

98 

49 

.60 

-15 

56 

84 

42 

.30 

.05 

39 

63 

32 

.26 

.01 

30 

52 

26 

Regressions 

The  compliance  rate  technique  described  above  could  potentially  misidentify  base  differ¬ 
ences  as  practitioner  group  differences.  An  alternative  technique  allows  us  to  control  for  any 
differences  in  compliance  across  bases;  it  compares  the  performance  of  the  three  groups  by 
estimating  the  contribution  of  each  individual’s  group  to  his  or  her  observed  compliance  rate 
using  a  regression  technique.  Since  we  can  also  estimate  the  contribution  of  the  individual’s 
base  location  to  his  or  her  compliance,  we  are  able  to  measure  group  differences  independent 
of  any  base  effects. 

Let  p,  equal  the  observed  compliance  rate  for  the  ith  practitioner  for  n  visits  where  a 
particular  criterion  applies.  Linear  regressions  using  p,  as  the  dependent  variable  are  inappro¬ 
priate  because  they  can  give  predictions  outside  the  limited  range  of  pt  (between  zero  and  one). 
In  our  case,  where  the  p,’s  are  often  close  to  either  zero  or  one,  this  problem  is  serious.  By 
estimating  a  transformation  of  the  p,  the  logit  or  logarithm  of  the  odds  <ln(p/  1  —  Pj))  our 
predictions  are  limited  to  the  appropriate  range.  For  each  criterion,  therefore,  we  estimate  the 
following  relationship: 

In  =  0n  +  0,(PA)  +  /3.JPCNP)  +  £3(CHAN)  +  04(DYESS)  +  &(FCH1LD)  +  U 


The  explanatory  variables  are  all  dummy  variables.  Thus  PA  =  1  if  the  practitioner  is  a 
PA,  0  otherwise;  and  PCNP  =  1  if  the  practitioner  is  a  PCNP,  0  otherwise.  Similarly,  CHAN, 
DYESS,  and  FCHILD  are  indicator  variables  for  each  of  the  bases.  We  chose  the  analysis  of 
variance  specification  for  these  variables;  each  equals  1  if  the  observation  is  from  the  indicated 
base,  - 1  if  from  Nellis,  and  0  otherwise.  For  example,  CHAN  =  1  if  Chanute  is  the  base,  0 
if  either  Dyess  or  Fairchild  is  the  base,  and  - 1  if  Nellis  is  the  base. 


With  this  specification,  j30  represents  the  average  (transformed)  compliance  rate  for  physi¬ 
cians  at  all  locations;  £,  and  02  represent  the  MD-PA  and  MD-PCNP  differences  in  perfor¬ 
mance.  fi3,  Pt  and  /3£  each  represent  the  differences  between  the  average  performance  at  all 
locations  and  the  performance  at  Chanute,  Dyess,  and  Fairchild,  respectively.  The  difference 
between  overall  performance  and  the  fourth  location,  Nellis,  is  represented  by  -(03  +  04  + 
0s>- 

We  estimate  the  coefficient  vector  0  by  finding  the  values  that  maximize  the  likelihood 
function.  Table  B.2  lists  the  coefficients’  estimates  and  the  associated  t-values.  These  t-values 
are  calculated  by  taking  the  square  root  of  twice  the  log  of  the  ratio  of  the  likelihoods  with 
and  without  the  variable  included  in  the  equation,  and  are  asymptotically  distributed  as 
standard  normal  variables.  The  table  also  shows  the  log  likelihood  ratio  at  the  solution. 

The  estimated  coefficients  on  the  PA  and  PCNP  variables  measure  the  MD-PA  and  MD- 
PCNP  differences  in  the  log  of  the  odds,  i.e.,  ln(p/l  -  p).  From  these  coefficients,  we  can 
estimate  differences  in  the  untransformed  compliance  rates  for  PAs  and  PCNPs  relative  to 
physicians,  as  follows: 

ip__s  sniPj (i - a> 

dxj  0j  Snj 

where  =  the  coefficient  on  x^;  n,  =  the  actual  counts  of  visits  to  the  ith  individual;  and  Pi 
=  the  estimated  compliance  rate  for  the  ith  individual.  Table  B.3  presents  these  untrans¬ 
formed  differences  for  each  criterion  tested.  Reading  from  the  first  line  of  the  table,  the  PA 
compliance  was  0.102  greater  than  the  MD  rates,  but  the  difference  was  not  statistically 
significant.  PCNP  compliance  was  0.05  less  than  MD  compliance.  Average  compliance  at 
Chanute,  controlling  for  different  practitioner  types,  was  0.158  higher  than  average  compli¬ 
ance  at  all  bases.  Nellis  compliance  was  significantly  lower  than  average.  We  conclude  that 
the  estimated  differences  are  significant  (represent  true  differences)  only  if  the  t-value  on  the 
original  estimated  coefficient  exceeds  2.24,  as  with  the  test  for  differences  in  proportion. 

Results  for  this  regression  technique  differ  from  the  simple  difference  in  proportions  test 
for  7  out  of  the  42  scoreboard  criteria.  Overall  conclusions  presented  in  the  scoreboard  would 
vary  little  if  the  regression  technique  were  used.  We  have  therefore  relied  on  the  simpler 
difference  in  proportions  test  for  the  discussion  of  results  and  conclusions  presented  in  Sec. 
Ill  and  App.  C. 


TESTS  ACROSS  ALL  CRITERIA 

In  addition  to  comparing  performance  on  individual  criteria,  we  were  interested  in  com¬ 
paring  performance  of  practitioners  based  on  the  full  set  of  criteria  we  examined.  Using  the 
42  nonredundant  criteria,  we  performed  sign  tests  to  show  whether  PAs  or  PCNPs  performed 
better  or  worse  on  a  significantly  large  number  of  criteria.  The  sign  test  is  a  nonparametric 
test  of  the  hypothesis  that  two  groups  perform  equally  well.  If  this  hypothesis  is  true,  one 
expects  a  probability  of  xh  that  one  group  scores  better  on  any  single  criterion.  If  the  data  show 
one  group  scoring  better  on  many  more  than  half  the  criteria,  it  suggests  that  the  group  is 
actually  performing  better  overall,  and  we  reject  the  hypothesis  that  the  groups  perform 
equally.  The  p-value  of  a  given  result,  i.e.,  pattern  of  scores,  indicates  the  probability  that  the 
result  could  occur  even  if  the  two  groups  do  not  differ  The  smaller  the  p-value,  the  more  likely 
the  two  groups  differ. 

Table  B.4  shows  the  results  of  the  sign  tests  comparing  the  MD  group  with  both  the  PA 
and  PCNP  groups.  Since  there  is  a  5  percent  or  less  chance  that  these  results  would  occur  if 
PAs  and  MDs  were  equal  in  performance,  we  have  strong  evidence  to  suggest  the  PAs  actually 
outperformed  MDs  in  compliance  with  these  process-of-care  criteria. 


S3 


We  alao  used  the  sign  test  to  test  whether  bases  complied  at  the  same  level.  Here  we  used 
the  estimated  coefficients  on  bases  from  the  logit  regressions.  A  positive  sign  on  the  coefficient 
indicated  that  the  base  was  better  than  average  for  desirable  criteria,  and  a  negative  sign 
indicated  the  base  was  better  for  undesirable  criteria.  Table  B.5  shows  the  results  of  these 
tests.  While  Fairchild  seems  to  have  performed  better  than  average  on  more  than  half  the 
criteria,  the  p-values  are  all  above  10  percent. 


Table  B.2 

Maximum  Likelihood  Estimates  op  Regression  Coefficients 

(with  estimated  t-statistics) 


B,  b2  83  e4  e5  B0 

Criterion 

Number  PA  PCNP  CHAU  DYESS  FCHILD  INTERCEPT  NELLIS  LLK 


Desirable  Diagnostic  Actions 

-0.023  -0.273  -0.464  6.4?4 


2. 

0.438 

<1.56) 

-0.213 

(-0.30) 

0.681 

(2.22) 

-0.193 

(-0.70) 

3. 

-0.430 

(-1.84) 

-0.948 
(-2.  92) 

0.488 

(1.73) 

-0.487 

(-2.87) 

4. 

-0.574 

(-2.02) 

-0.706 

(-1.45) 

-0.489 

(-2.03) 

0.244 

(1.29) 

6. 

2.265 

(6.91) 

1.405 

(3.08) 

-0.035 

(-0.16) 

0.025 

(0.10) 

7. 

-0.976 

(-1.19) 

0.846 

(0.67) 

0.084 

(0.21) 

0.046 

(0.09) 

10. 

1.415 

(1.65) 

0.804 

(0.76) 

0.449 

(0.90) 

-1.007 

(-2.11) 

11. 

0.223 

(1.21) 

-0.130 

(-0.50) 

-0.280 

(-2.07) 

0.429 

(3.34) 

14. 

1.583 

(1.56) 

0.317 

(0.26) 

-1.907 

(-2.17) 

0.010 

(0.10) 

15. 

1.037 

(0.77) 

1.604 

(0.82) 

2.410 

(0.85) 

1.416 

(0.50) 

16. 

0.265 

(0.23) 

-0.502 

(-0.34) 

-0.561 

(-0.68) 

0.216 

(0.29) 

(-0.09) 

(-1.34) 

(-2.25) 

0.962 

(5.17) 

1.036 

(4.95) 

-0.964 

(-6.14) 

38.972 

0.784 

(3.83) 

0.107 

(0.77) 

0.539 

(-2.92) 

14.058 

0.978 

(2.89) 

0.322 

(1.48) 

-0.968 

(-3.45) 

35.017 

0.083 

(0.16) 

2.090 

(2.74) 

-0.212 

(-0.46) 

2.973 

1.042 

(1.57) 

-0.626 

(-0.78) 

-0.484 

(-1.08) 

4.956 

0.327 

(2.38) 

-0.031 

(-0.20) 

-0.521 

(-3.92) 

14.099 

1.634 

(2.43) 

-2.030 

(-2.33) 

0.173 

(0.28) 

6.756 

* 

-3.109 

(-1.04) 

1.594 

(0.57) 

2.436 

0.690 

(0.62) 

-0.312 

(-0.33) 

-0.345 

(-0.47) 

0.405 

17. 


-0.318  -0.240  -0.018  -0.035  -0.064  0.376 

(-1.05)  (-0.57)  (-0.08)  (-0.17)  (-0.26)  (1.47; 


0.118 

(0.43) 


0.580 
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Table  B.2— continued 


Criterion 

Number 

6i 

PA 

**2 

PCNP 

<*3 

CHAN 

8* 

DYESS 

*5 

KCH1LD 

B0 

INTERCEPT 

-(63+64+65 f 

NELLIS 

LLK 

Desirable  Therapeutic  Actions 

18. 

-0.691 

* 

-0.375 

0.513 

-2.179 

2.177 

0.408 

6.827 

(-1.03) 

(-1.21) 

(1.16) 

(-1.74) 

0.41) 

(0.88) 

19. 

2.656 

0.947 

-1.537 

-0.523 

* 

2.691 

-2.092 

5.922 

(2.22) 

(0.97) 

(-0.59) 

(-0.20) 

(1.04) 

(-0.81) 

21. 

0.310 

0.255 

-0.608 

0.178 

0.466 

0.181 

-0.376 

2.612 

(0.96) 

(0.521) 

(-1.90) 

(0.74) 

(1.52) 

(0.73) 

(-0.15) 

22. 

0.168 

-1.597 

-2.341 

0.677 

0.370 

-0.761 

-0.812 

7.120 

(0.33) 

(-1.80) 

(-0.53) 

(1.76) 

(1.24) 

(-1.69) 

(-2.09) 

24. 

1.040 

-0.082 

-1.658 

-2.125 

* 

3.174 

-1.918 

7.090 

(1.91) 

(-0.13) 

(-0.57) 

(-0.73) 

(1.08) 

(-0.65) 

25. 

1.901 

2.213 

-0.389 

-0.174 

1.928 

0.422 

-1.365 

16.285 

(4.18) 

(3.07) 

(-1.02) 

(-0.43) 

(2.42) 

(1.14) 

(-3.30) 

26. 

1.726 

* 

-0.419 

0.944 

0.658 

0.303 

-0.689 

5.69 

(1.24) 

(-1.01) 

(2.18) 

(0.50) 

(0.22) 

(-1.22) 

28. 

-2.324 

-1.897 

0.823 

-0.339 

-0.623 

1.721 

0.139 

9.501 

(-3.23) 

(-2.37) 

(2.36) 

(-0.86) 

(-1.45) 

(2.64) 

(0.35) 

30. 

1.078 

-0.795 

-2.901 

-2.004 

* 

2.175 

0.756 

3.306 

(0.83) 

(-0.41) 

(1.00) 

(-0.70) 

(0.73) 

(-0.26) 

31. 

2.301 

3.588 

-0.205 

-1.102 

-0.980 

-1.654 

0.229 

5.883 

(1.97) 

(1.89) 

(-0.33) 

(-1.31) 

(-0.93) 

(-1.71) 

(2.32) 

Undesirable  diagnostic  Actions 

33. 

-0.670 

-1.657 

0.327 

-0.751 

0.317 

-4.941 

0.107 

6.852 

(-2.14) 

(-2.22) 

(1.39) 

(-2.08) 

(1.04) 

(-20.51) 

(0.39) 

35. 

-0.029 

-0.113 

0.036 

-0.222 

0.395 

-2.699 

-0.208 

17.345 

(-0.31) 

(-0.83) 

(0.53) 

(-2,87) 

(5.55) 

(-35.592) 

(-2.91) 

36. 

1.320 

2.404 

1.567 

* 

2.256 

-5.098 

1.904 

10.818 

(1.22) 

(2.14) 

(0.58) 

(0.82) 

(-1.79) 

(0.94) 

38. 

0.203 

0.572 

0.489 

0.351 

-0.800 

-1.064 

-0.041 

1.660 

(0.31) 

(0.65) 

(1.37) 

(0.90) 

(-1.43) 

(-1.68) 

(-0.11) 

39. 

1.315 

1.072 

0.499 

0.266 

-0.316 

-1.444 

-0.449 

17.437 

(3.86) 

(2.61) 

(3.11) 

(1.29) 

(-1.66) 

(-4.51) 

(-2.46) 
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Table  B.2 — continued 


Criterion 

Number 

61 

PA 

PCNP 

63 

CHAN 

s« 

DYESS 

S5 

FCHILD 

INTERCEPT 

-(By*V05)a 

NELLIS 

LLK 

Desirable  Disposition  Action 

56. 

0.381 

-1.166 

2.578 

0,066 

0.116 

1.495 

-0.440 

15.558 

U.68) 

(4.18) 

(1.10) 

(0.39) 

(0.62) 

(10.89) 

(-2.83) 

57. 

-0.340 

-0.425 

0.054 

-0.080 

0.141 

1.031 

-0.116 

6.927 

(-2.95) 

(-2.24) 

(0.45) 

(-0.83) 

(1.51) 

(11.68) 

(-1.35) 

59. 

-0.479 

0.917 

-0.091 

0.489 

0.531 

0.455 

-0.929 

7.981 

(-1.25) 

(1.33) 

(-0.29) 

(1.50) 

(1.58) 

(1.78) 

(-3.19) 

60. 

-0.199 

-0.896 

-0.423 

0.518 

0.379 

0.291 

-0.474 

7.746 

(-0.75) 

(-1.83) 

(-1.84) 

(2.24) 

(1.04) 

(1.45) 

(-2.29) 

61. 

1.167 

0.104 

-0.105 

-0.110 

-0.041 

0.582 

0.256 

3.808 

(2.34) 

(0.14) 

(-0.28) 

(-0.28) 

(-0.09) 

(1.62) 

(0.54) 

aThe  t-values  for  the  Nellis  effect  are  calculated  from  the  estimated  variance- 


covariance  matrix  of  the  coefficient  estimates. 


Table  B.3 


Estimated  Differences  in  Untransformed  Compliance  Rates 
PA  and  PCNP  Compared  with  MD;  Bases  Compared  with  Overall  Average 


Criterion 


Number 

PA 

PCNP 

CHAN 

DYESS 

FCHILD 

NELLIS 

Desirable  Diagnostic  Actions 

2. 

0.102 

-0.050 

0.158 

-0.045 

-0.005 

-0.108* 

3. 

-0.089 

-0.196* 

0.101 

-0.100* 

0.199* 

-0.199* 

4. 

-0.132 

-0.162 

-0.113 

0.056 

0.180* 

-0.124* 

6. 

0.287* 

0,178* 

-0.004 

0.003 

0.124* 

-0.123* 

7. 

-0.140 

0.122 

0.012 

0.007 

0.012 

-0.031 

10. 

0.293 

0.16? 

0.093 

-0.208 

0.216 

-0.100 

11. 

0.054 

-0.031 

-0.067 

0.103 

0.069 

-0.125* 

14. 

0.228 

0.046 

-0.275 

0.014 

0.236 

0.025 

15. 

0.195 

0.302 

0.453 

0.266 

** 

0.300* 

16. 

0.062 

-0.118 

-0.132 

0.051 

0.162 

-0.081 

17. 

-0.079 

-O.059 

-0.005 

-0.009 

-0.016 

0.029 

Desirable 

Therapeutic  Actions 

18. 

-0.095 

** 

0,023 

0.145 

-0.225 

0.056 

19. 

0.140 

0.051 

-0.082 

-0.028 

** 

-0.112 

21. 

0.072 

0.059 

-0.142 

0.041 

0.109 

-0.009 

22. 

0.033 

-0.315 

-0.046 

0.133 

0.073 

-0.160 

24. 

0.107 

-0.008 

-0.171 

-0.219 

A* 

-0.198 

25. 

0.272* 

0.317* 

-0.056 

-0.025 

0.276* 

-0.196* 

26. 

0.127 

A* 

-0.043 

0.057 

0.036 

-0.051 

28. 

-0.487* 

-0.397* 

0.173* 

-0.071 

-0.130 

0.029 

30. 

0.152 

-0.112 

-0.409 

-0,283 

A* 

-0.107 

31. 

0.409 

0.637 

-0.036 

-0,196 

-0.174 

0.406* 

Undesirable  Diagnostic  Actions 

33. 

-0.003 

-0,008 

0.001 

-0,003 

0.001 

0.000 

35. 

-0.002 

-0.006 

0.002 

-0.013* 

0.023* 

-0.012* 

36. 

0.162 

0.294* 

0.123 

** 

0.207 

0.233 

38. 

0.044 

0.124 

0.10b 

0.07b 

-0.173 

-0.009 

39. 

0.298* 

0.243* 

0.113* 

0.060 

-0.072 

-0.108* 

Undesirable  Therapeutic  Actions 

40. 

-0.008 

-0.003 

-0.12b 

0. 049 

0.032 

0.045 

41. 

-0.066* 

-0.040 

0.131 

-0.058* 

0.012 

0.015 

42. 

-0.027 

0.002 

-0.044 

-0.002 

0.003 

0.044 

43. 

0.036 

0.065 

-0.012 

0.012 

0.024 

-0.024 

47. 

-0.006 

0.001 

0.010 

-0.007 

0,006 

-0.008 

48. 

-0.019 

0 .027 

** 

-0.525 

-0.211 

-0.215 

51. 

-0.003 

** 

** 

0.003 

0.003 

0.005 

52. 

-0.141 

-0.327* 

0.009 

0.015 

-0.113 

0.089 

54. 

-0.007 

-0.0003 

-0.017 

-0.021 

-0.019 

** 

55. 

-0.008 

** 

-0.038 

** 

A* 

0.035 

Desirable 

Disposition  Actions 

56. 

0.058 

-0.179* 

0.040 

0,010 

0.018 

-0.067* 

57. 

-0.072* 

-0.090* 

0.C12 

-0.017 

0.030 

-0.025 

59. 

-0.106 

0.202 

-0.020 

0.108 

0.127 

-0.205* 

60. 

-0.047 

-0.212 

-0.100 

0,122* 

0.090 

-0.112* 

61. 

0.203* 

0,018 

-0.018 

-0.019 

-0.007 

0.045 

*  -  Coefficient  significantly  different  from  0. 

**  -  Coefficient  undefined  due  to  OX  or  100X  compliance  In 


group 
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Table  B.4 


Sign  Tests  on  the  Difference  Between  MD  and  PA 
Performance  and  MD  and  PCNP  Performance 

Based  on  Group  Performance  Rates 


Result 

PAs 

PCNPs 

Extender  performed  better 

28  criteria 

21  criteria 

Ties 

1 

1 

MD  performed  better 

13 

20 

Significance 

p  =  0.05 

p  >  0.25 

NOTE;  Although  these  results  rely  on  group  com¬ 
pliance  rates  as  presented  in  App.  C,  we  also  did 
the  sign  test  based  on  the  estimated  coefficients 
for  the  practitioner  variables  in  the  logit  regres¬ 
sions.  The  results  were  identical  to  those  above. 


Table  B.5 

Sign  Tests  on  the  Difference  Between  Base 
Performance  and  Overall  Mean  Performance 

Based  on  Logit  Regression  Coefficients 


Criteria 

Chanute 

Dyess 

Fairchild 

Uellis 

Criteria  where  base 
better  than  average 

15 

21 

26 

15 

Criteria  where  base 
less  than  average 

26 

20 

15 

26 

Significance 

0.10<p<0.25 

p>0. 25 

0.10<p<0,25 

0.10<p<0.25 

Appendix  C 


DISCUSSION  OF  SELECTED  CRITERIA  OF 
QUALITY 


This  appendix  presents  the  complete  list  of  condition-specific,  technical-process-of-care 
criteria  upon  which  we  base  our  conclusions.  We  report  62  criteria  in  all,  but  will  not  discuss 
all  of  them  singly.  We  wish  neither  to  present  a  criterion-by-criterion  medical  justification  for 
inclusion,  nor  to  discuss  the  possible  meaning  of  the  results  of  each  one.  However,  certain 
criteria  merit  particular  attention  because  they  illustrate  points,  provide  surprising  or  note¬ 
worthy  results,  show  contrasts,  or  require  explanation  or  speculation. 


DESIRABLE  DIAGNOSTIC  ACTIONS 

We  start  with  desirable  diagnostic  actions  (Table  C.l).  Two  criteria  pertain  to  diabetes 
mellitus,  one  written  to  be  slightly  more  lenient  than  the  other;  the  results  are  virtually 
identical  for  both.  It  is  not  surprising  that  the  percentage  of  ordering  one  or  the  other  of  these 
tests  is  as  modest  as  it  is,  when  it  is  considered  that  all  visits  are  included,  not  merely  the  first 
visit  for  the  problem.  It  would  not  ordinarily  be  necessary  to  order  a  blood  sugar  check  on  each 
visit  by  every  diabetic. 

We  come  next  to  the  recording  of  a  blood  pressure  measurement,  when  the  provider 
checked  off  "hypertension”  as  the  only  diagnosis  on  the  Patient  Contact  Record.  Both  first 
visits  and  return  visits  are  included  in  this  criterion.  The  level  of  compliance  for  all  three 
provider  groups  is  below  what  one  might  expect  (although  we  have  no  comparable  data  to 
know  whether  this  level  is  lower  than  that  found  for  groups  of  providers  in  other  settings). 
A  closer  look  showed  significant  differences  among  bases:  Two  of  the  four  bases  had  compli¬ 
ance  levels  of  40  to  60  percent,  while  the  other  two  had  compliance  levels  in  the  70  to  80  percent 
range.1  It  is  possible  that  blood  pressure  was  taken  and  recorded  on  the  patients’  charts,  but 
that  the  Patient  Contact  Record  was  not  checked  off  properly.  However,  that  is  conjecture. 

It  would  be  possible  to  accept  this  result  as  an  indicator  of  possible  low  quality  by  all 
providers  at  the  two  bases  (or  even  at  all  four  of  the  bases)  and  to  conduct  a  chart  review  to 
try  to  ascertain  the  extent  of  the  apparent  problem.  However,  recall  that  we  are  primarily 
interested  in  the  comparative  performance  of  the  three  groups,  in  order  to  answer  the  basic 
question:  Are  PAs  and  PCNPs  measuring  up  to  the  quality  of  care  level  provided  by  physicians 
in  the  primary  medical  setting?  While  some  of  these  results,  such  as  this  one  dealing  with  blood 
pressure  measurement  when  there  is  hypertension,  might  serve  as  red  flags  that  mark  poten¬ 
tial  quality  problems  among  all  groups  of  providers,  that  issue  is  beyond  the  scope  of  our 
current  survey. 

Two  criteria  deal  with  the  situation  in  which  pharyngitis  (sore  throat)  is  the  only  specified 
diagnosis,  and  the  patient  is  making  his  or  her  first  visit  for  the  condition.  Both  are  presented 
because  different  analysts  might  prefer  one  criterion  over  the  other.  Although  the  percentages 
of  compliance  are  different,  the  results  and  the  ranking  are  the  same.  Both  the  PAs  and  the 
PCNPs  outperform  the  physicians,  whether  we  "require”  that  a  throat  culture  be  ordered  if 


'The  compliance  rates  of  all  practitioners  combined  may  differ  among  the  bases;  e.g.,  base  A  may  have  significantly  higher 
overall  compliance  than  base  B.  This  is  a  "base  effect.”  For  further  discussion,  see  App.  B. 

59 


60 


Table  C.l 


Desirable  Diagnostic  Actions 


Criterion  and 

MD 

PA 

PCNP 

St  at  1st ically 

Desirable  Action 

Only 

Only 

Only 

Signif leant ? 

1 

Diabetes  mellitus;  blood  sugar 

38* 

51% 

30% 

no 

ordered 

n-120 

n-102 

n=10 

2  ° 

Diabetes  mellitus;  blood  sugar 

38% 

52% 

30% 

no' 

or  urinalysis  ordered 

n-120 

n-102 

n-10 

no 

3C 

Hypertension  (only);  blood 

66% 

61% 

55% 

pressure  taken 

n-123 

n-35U 

n-84 

Ischemic  or  other  heart  disease; 

53a 

4  3% 

32% 

no' 

heart  auscultated 

n-2  36 

n-73 

n-22 

5 

Pharyngitis  (only,  first  visit); 

43% 

84% 

80% 

PA  >  MD 

throat  culture  ordered 

n-125 

n-223 

n-73 

PCNP  >  MD 

6C 

Pharyngitis  (only,  first  visit); 

58% 

92% 

88% 

PA  >  MD' 

throat  culture  ordered  or 
penicillin  prescribed 

n-125 

n-223 

n-73 

PCNP  >  MD 

f 

Urinary  tract  infection  (only, 

90% 

75% 

95% 

no 

first  visit);  urine  culture  or 
urinalysis  ordered 

n-19 

n-69 

n*20 

8 

Urinary  tract  infection  (only. 

84% 

71% 

95% 

no 

first  visit);  urinalysis  ordered 

n-19 

n-69 

n-20 

9 

Urinary  tract  infection  (only. 

74% 

62% 

85% 

no 

first  visit);  urine  culture 
ordered 

n-19 

n-69 

n-20 

10a 

Urinary  tract  infection,  female, 

38% 

t>52 

50% 

no 

16+  years  (return  visit);  uri¬ 
nalysis  or  urine  culture  ordered 

n-8 

n-46 

n-12 

ne 

Lung  diseases;  lungs  auscultated 

51% 

52% 

50% 

L 

no 

n-175 

n-428 

n-106 

12 

Pneumonia  (first  visit);  chest 

67% 

80% 

67% 

no 

x-ray  or  sputum  culture  ordered 

n-3 

n-20 

n-9 

12 

Heart  diseases  (only,  first 

24% 

27% 

33% 

no 

visit);  ECG  ordered 

n-17 

n-11 

n-3 

K.d 

"Pelvic  exam  indicated”  group; 

13% 

36% 

25% 

no 

pelvic  exam  done 

n-15 

n-25 

n-8 

t 

no 

if 

Conorrhea  (first  visit); 

20% 

46% 

20% 

VDRL  ordered 

n-5 

n-13 

n-5 

16^ 

Gonorrhea  (first  visit);  culture 

40% 

46% 

4U% 

no 

or  gram  stain  ordered 

n-5 

n-13 

n-5 

if 

Earache  (reason  for  visit) 

59% 

52% 

53% 

no 

(children  only);  any  type  of 
otitis  diagnosed 

n-70 

n-135 

n-36 

MOTE:  "First  visit"  indicates  a  first  visit  for  the  specified  condi¬ 
tion.  "Only"  Indicates  that  only  one  diagnostic  category  was  marked  on 
that  visit,  "n"  equals  the  number  of  visits  (not  necessarily  the  number 
of  patients). 

^Criterion  used  In  sign  tests  and  PA-MD  comparisons,  but  not  in  PCNP-MD 
comparisons. 

Regression  results  indicated  significant  differences  among  bases. 

See  App.  B. 

^Criterion  used  in  sign  tests  and  all  summary  comparisons. 

Criterion  used  in  sign  tests  only. 


the  provider  makes  the  sole  diagnosis  of  pharyngitis,  or  we  give  the  provider  the  option  either 
of  ordering  a  throat  culture  or  prescribing  penicillin  (or  erythromycin).2  Most  experts  would 
argue  that  if  the  provider  is  to  make  the  diagnosis  of  pharyngitis  (sore  throat),  and  sore  throat 
is  the  only  problem  checked  off  by  the  provider  (that  is,  runny  nose,  fever,  or  cough  was  not 
checked  in  addition),  then  a  throat  culture  should  be  obtained  to  make  certain  that  the  throat 
has  not  been  colonized  with  a  bacterium  for  which  treatment  with  an  antibiotic  is  essential. 
All  three  groups  of  providers  saw  large  numbers  of  first-visit  patients  with  pharyngitis  alone. 
The  two  types  of  extenders  outperformed  the  physicians  in  obtaining  throat  cultures.  (As  for 
base  effect,  all  providers  at  one  of  the  four  bases  performed  significantly  better,  and  providers 
at  a  second  base  performed  significantly  worse.) 

We  might  ask:  Are  the  43, 84,  and  80  percent  figures  acceptable  in  absolute  terms?  We  are 
fortunate  in  having  comparable  data  from  the  civilian  sector  for  this  criterion.  In  1976,  Kane 
and  co-workers  fielded  an  encounter  form  among  physician  preceptors  and  their  MEDEX  (a 
type  of  PA)  in  private  offices  throughout  the  state  of  Utah.  The  data  revealed  that  the 
physicians  obtained  throat  cultures  for  pharyngitis  58  percent  of  the  time,  while  the  MEDEX 
did  so  68  percent  of  the  time  (Kane,  Olsen,  and  Castle,  1976).  These  percentages  show  that  Air 
Force  performance  levels  are  comparable  to  levels  reached  in  the  civilian  sector.3  Some 
observers  would  claim  that  throat  cultures  are  unnecessary  under  certain  circumstances; 
indeed,  an  appropriate  antibiotic  should  be  immediately  prescribed.  We  therefore  wrote  a 
second  criterion  for  pharyngitis,  thus  offering  the  option  of  either  culturing  the  throat  or 
prescribing  penicillin.  It  is  to  be  noted  that  the  rank  order  of  compliance  with  the  criterion 
by  the  three  groups  of  providers  is  unchanged,  and  the  percentages  are  similar. 

We  turn  now  to  urinary  tract  infection  (UTI).  The  first  three  criteria  provide  alternate 
retrospective  requirements  when  the  provider  has  checked  UTI  as  the  sole  diagnosis  on  the 
patient’s  first  visit.  For  this  condition,  as  for  all  the  others,  the  condition  was  marked  off  on 
the  Patient  Contact  Record  by  the  provider  at  the  actual  time  of  visit.  The  three  criteria  are 
provided  because  reasonable  observers  might  disagree  on  what  exactly  should  usually  be 
ordered  when  the  diagnosis  of  UTI  is  being  entertained.  It  is  therefore  important  to  note  that 
no  statistical  significance  is  found,  no  matter  which  of  the  three  alternate  forms  of  the  criterion 
is  used,  and  that  the  rank  order  is  the  same  for  all  three.  High  percentage  compliance  is  found 
for  all  three  groups  of  providers.  Furthermore,  we  note  no  base  effect. 

The  final  criterion  concerning  UTIs  is  quite  different.  It  specifies  that  adult  women  coming 
for  a  return  visit  for  UTI  should  have  a  urinalysis  or  urine  culture  done.  Such  follow-up 
diagnostic  action  would  be  taken  presumably  to  confirm  the  efficacy  of  treatment.  Again,  there 
is  no  statistically  significant  difference  in  performance,  although  the  rank  order  of  compliance 
with  the  criterion  has  changed. 

The  criterion  that  the  lungs  should  be  auscultated  if  a  patient  comes  in  with  a  "lung 
disease”  is  based  on  the  "lung  disease”  group  of  diagnoses  (see  Table  C.6  at  the  end  of  this 
appendix).  In  a  large  number  of  visits,  an  amazingly  similar  percentage  of  auscultation  was 
reported  by  the  three  groups  of  providers. 

Results  for  the  remaining  criteria  in  this  desirable  diagnostic  action  criterion  class  are  also 
consistent  with  the  conclusion  that  PEs  are  performing  at  a  level  of  care  equal  to  that  of 
physicians. 
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DESIRABLE  THERAPEUTIC  ACTIONS 

The  next  criterion  class  examines  performance  according  to  criteria  of  desirable  therapeu¬ 
tic  actions  (Table  C.2).  The  first  criterion  states:  If  the  patient  had  acne  and  was  treated  with 
an  antibiotic,  then  that  antibiotic  was  tetracycline.  For  89  percent  of  antibiotic-treated  acne 
patients  seen  by  physicians,  the  physician  wrote  a  prescription  for  tetracycline;  the  corre¬ 
sponding  figures  were  78  percent  for  PAs  and  100  percent  for  PCNPs.  The  difference  is  not 
statistically  significant. 

If  a  patient  has  acute  sinusitis,  and  is  being  seen  for  the  first  time  for  this  condition,  then 
a  medication  should  be  prescribed.  This  was  done  very  often  by  all  three  groups,  and  was  done 
significantly  more  often  by  PAs  than  by  physicians.  (The  difference  between  PCNPs’  perfor¬ 
mance  and  physicians’  performance  is  not  statistically  significant.)  If  we  make  the  criterion 
more  precise  (and  also  more  arguable)  by  requiring  prescription  of  an  antibiotic,  then  the 
percentages  change,  but  the  conclusion  does  not. 

It  is  generally  conceded  that  if  a  medication  is  prescribed  for  asthma,  the  sine  qua  non 
is  a  bronchial  dilator.  It  was  prescribed  equally  often  by  all  three  provider  groups. 

The  criterion  for  hypertension  states  that  if  a  patient  is  seen  on  a  return  visit  for  hyperten¬ 
sion  (implying  that  the  diagnosis  has  been  confirmed),  then  an  antihypertensive  agent  and/or 
a  diuretic  was  prescribed.  The  low  percentage  compliance  with  this  criterion,  coupled  with  the 
low  percentage  compliance  for  blood  pressure  measurement,  suggests  that  there  well  may  be 
a  problem  in  management  of  hypertension  at  these  demonstration  bases.  For  our  purposes, 
it  is  sufficient  to  point  out  that  the  apparent  problem  seems  to  be  spread  among  all  three 
groups  of  providers.  Chart  review,  which  was  beyond  the  scope  of  our  survey,  would  be  one 
means  of  assessing  how  genuine  the  problem  is  and  how  severe  it  might  be.  Note,  also,  that 
the  significant  difference  among  bases  found  for  blood  pressure  measurement  was  not  found 
for  the  criterion  concerning  blood  pressure  management. 

For  otitis  media  (inflammation  of  the  middle  ear),  a  common  problem  in  children,  a 
prescription  should  have  been  written  during  the  first  visit,  when  the  condition  was  checked 
off  by  the  provider.  We  present  two  criteria  dealing  with  the  prescription  of  an  antibiotic;  the 
results  are  virtually  the  same  no  matter  which  criterion  is  applied.  They  show  that  prescrip¬ 
tions  are  written  equally  often  by  all  three  provider  groups. 

What  should  we  make  of  the  fact  that  none  of  the  percentages  are  100  percent?  Should 
100  percent  of  patients  with  infectious  otitis  media  have  a  prescription  written  on  the  first 
visit?  It  would  likely  be  the  consensus  of  any  group  of  physicians  that  this  is  so.  However,  there 
are  good  reasons  why  we  should  not  expect  a  100  percent  performance.  For  one  thing,  it  is 
perfectly  possible  that  the  provider  occasionally  forgot  to  check  a  box,  or  checked  the  wrong 
box  on  the  Patient  Contact  Record.  Secondly,  it  is  perfectly  possible  that  the  provider  was 
about  to  write  a  prescription  when  the  child’s  parent  said,  "Oh,  don’t  bother  to  write  a 
prescription;  we  have  some  of  that  medicine  left  at  home  from  the  last  time.”  For  these  and 
other  reasons,  it  is  unrealistic  to  expect  100  percent  compliance  with  any  criterion.  All  three 
percentage  levels  are  very  high,  and  suggest  that  appropriate  prescriptions  were  being  written 
for  infectious  otitis  media  in  virtually  all  cases  by  all  group*  of  providers.  Note  that  we  also 
included  a  criterion  concerning  the  prescription  of  decongestants,  for  both  infectious  and 
noninfectious  (serous)  otitis  media.  It  is  considered  appropriate  therapy  to  prescribe  a  decong¬ 
estant  for  children  with  otitis  media;  the  PEs  appear  to  have  outperformed  the  physicians  in 
this  regard,  a  finding  not  affected  by  the  presence  of  significant  differences  in  performance 
among  bases. 

It  is  generally  accepted  that  if  a  UTI  is  diagnosed,  some  medicine  should  be  prescribed  at 
the  time  of  initial  presentation  with  the  infection.  Here  is  a  situation  where  the  physicians 
appear  to  be  outperforming  the  extenders;  at  least,  if  the  criterion  requires  the  prescription 
of  an  antibiotic,  physicians  are  performing  statistically  significantly  better  than  either  group 


Table  C.2 

Desirable  Therapeutic  Actions 


MU 

PA 

PCNP 

Criterion 

Only 

Only 

Only 

Iff' 

Acne  treated  with  antibiotic; 

89/. 

78% 

100% 

antibiotic  is  tetracycline 

n*2b 

n=ll  7 

n-17 

19J 

Acute  sinusitis  (first  visit); 

77/. 

98% 

9  3% 

any  medication  prescribed 

n*l7 

n*b(J 

n*2  7 

20 

Acute  sinusitis  (first  visit); 

29% 

73% 

59% 

antibiotic  prescribed 

n*l  7 

n=bO 

n*2  7 

21 u 

Asthma;  bronchial  dilator 

57% 

627. 

62% 

prescribed 

n=*75 

n=89 

n=2b 

22“ 

hypertension  (return  visit); 

35% 

3b/. 

JU% 

ant ihyper tensive  and/or  diuretic 
prescribed 

n“2  3 

n-90 

n-20 

24 

Otitis  media  ( infect ious) ( f i rst  visit); 

88 .. 

94% 

80% 

antibiotic  proscribed 

n*51 

n«iuu 

n=30 

2 S 

Otitis  media  (infectious) (first  visit); 

82% 

92% 

80% 

penicillin,  erythromycin,  or  sulfa  prescribed 

n*51 

n-lUO 

n=  JO 

25“ 

Otitis  media  (infectious) (first  visit); 

57% 

84% 

90% 

decongestant  prescribed 

n*Sl 

n»100 

n  *  30 

26^ 

Otitis  media  (noninfectious) (f irst  visit); 

7S~ 

90  r. 

100% 

decongestant  prescribed 

n*=4 

n»132 

n=2  7 

27 

Urinary  tract  Infection  (only,  first  visit); 

9U% 

64% 

85% 

medication  prescribed 

n=19 

n*b9 

n»20 

2«“ 

Urinary  tract  infection  (only,  first  visit); 

b4/: 

42% 

40% 

antibiotic  prescribed 

n-19 

n-b9 

n*20 

29 

Laceration  with  suturing; 

21:. 

25% 

0% 

inrounlzation  given 

n-14 

n*8 

n=2 

40“ 

Gonorrhea  (first  visit); 

no.; 

7  7% 

80% 

antibiotic  prescribed 

n»5 

n*  l  3 

n*5 

31“ 

Obesity  lonly) ;  counseling  done 

Jb* 

b2% 

80.. 

n-1 1 

n*2 1 

n*5 

a 

Criterion  used  in  sign  tests  and  PA-MU  comparisons 

,  but 

not  in 

PCNP-MD  comparisons. 

“Criterion  used  In  sign  tests  only. 

Criterion  used  in  sign  tests  and  all  Summary  comparisons. 

"^Regression  results  indicated  significant  differences  among 
bases.  See  App.  B. 


Statistically 


PA  >  MI)  * 
PCNP  »  MD 


MU  ■  PA  ' 
Ml)  >  PCNP 
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of  extenders.  Although  it  is  possible  that  some  of  the  extenders  were  awaiting  the  results  of 
a  urine  culture  before  prescribing  medication,  it  is  likely  that  these  compliance  levels  point 
to  an  area  where  extenders’  prescribing  habits  could  be  improved. 

This  is  perhaps  a  good  juncture  to  note  that  it  should  not  be  astonishing  or  upsetting  to 
see  PEs  generally  performing  at  the  same  or  higher  levels  for  the  common  conditions  that  they 
have  been  trained  to  treat.  One  of  the  main  goals  behind  the  employment  of  highly  trained 
extenders  such  as  PAs  and  PCNPs  is  to  free  physicians  for  management  of  patients  with  more 
complex  and  challenging  problems.  The  summary  table  in  Sec.  Ill  showed  that,  insofar  as  we 
can  determine,  PAs  and  PCNPs  seem  to  be  measuring  up  to  physicians;  and  there  is  evidence 
from  the  sign  tests  on  all  criteria  that  PAs  may,  in  fact,  perform  at  a  higher  level  on  these 
criteria. 


UNDESIRABLE  DIAGNOSTIC  ACTIONS 

The  undesirable  diagnostic  actions  criteria  (Table  C.3)  permit  us  to  search  for  evidence  of 
possible  "overordering”  by  various  groups  of  providers.  Notice  that  because  the  action  is 
undesirable,  the  lower  the  percentage  the  better  care  is  likely  to  be.  The  first  four  criteria  are 
presented  as  fractions,  and  are  rather  difficult  to  follow.  The  first  and  third  fractions  are: 

Number  of  procedures  of  a  specified  type  'not  needed’ 

Number  of  procedures  of  a  specified  type  performed 

The  question  addressed  by  the  first  criterion  in  this  group  is:  Were  electrocardiograms 
(EKGs)  ordered  for  inappropriate  diagnoses?  To  approach  this  question,  we  first  made  a  list 
of  diagnoses  for  which  EKGs  are  likely  to  be  appropriate.  We  then  counted  the  number  of 
EKGs  ordered  for  diagnoses  that  did  not  appear  on  the  "appropriate”  list.  Of  137  electrocardio¬ 
grams  ordered  by  physicians,  14  percent  were  ordered  for  diagnoses  for  which  it  would  be 
unusual  to  need  an  EKG.  For  PAs,  21  percent  of  the  109  EKGs  they  ordered  were  for  diagnoses 
for  which  electrocardiograms  were  unlikely  to  be  appropriate.  For  PCNPs,  8  percent  of  the 
(only)  24  EKGs  they  ordered  were  for  diagnoses  for  which  it  would  be  unusual  to  need  an  EKG. 
The  differences  are  not  statistically  significant. 

For  the  third  criterion,  we  again  started  by  making  a  list  of  diagnoses  for  which  (in  this 
case)  x-rays  are  likely  to  be  appropriate.  We  then  counted  the  number  of  x-rays  ordered  for 
diagnoses  that  did  not  appear  on  the  "appropriate”  list.  For  all  three  provider  groups,  a  large 
percentage  of  the  large  number  of  x-rays  was  ordered  for  diagnoses  not  on  the  "appropriate” 
list.  This  may  suggest  that  the  list  was  faulty,  or  that  in  fact  there  is  a  substantial  amount  of 
misordering  of  x-rays.  The  situation  could  be  examined  on  a  local  basis. 

The  second  and  fourth  criteria  again  concern  inappropriate  ordering,  but  here  the  fraction 
is: 


Number  of  visits  when  procedures  were  ordered  for  diagnoses  not  on  the  "procedures 

_ appropriate”  list _ 

Total  number  of  visits  for  diagnoses  not  on  the  "procedures  appropriate”  list 

Again,  we  made  a  list  of  diagnoses  for  which  taking  an  EKG  is  likely  to  be  appropriate.  We 
then  looked  at  the  number  of  visits  for  all  other  (EKG  "not  needed”)  diagnoses,  and  computed 
how  often  EKGs  were  ordered  during  visits  for  these  "inappropriate”  diagnoses.  For  thou¬ 
sands  of  visits,  all  three  provider  groups  rarely  ordered  EKGs  for  inappropriate  diagnoses, 
although  physicians  did  order  them  (statistically  significantly)  more  inappropriately  than  did 
the  PCNPs.  For  x-rays,  again  with  a  large  number  of  observations,  all  three  provider  groups 
infrequently  ordered  x-rays  for  inappropriate  diagnoses. 
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Table  C.3 


Undesirable  Diagnostic  Actions 


Criterion 

MD 

Only 

PA 

Only 

PCNP 

Only 

Statistically 

Significant? 

32 

No.  of  EKGs  on  young  adults  without  "EKG  condition"0 

142 

21% 

8% 

no 

Total  no.  of  EKGs  ordered 

n-137 

n«109 

n*24 

33^ 

ilo.  of  EKGs  on  young  adults  without  "EKG  condition"* 

0.7% 

0.4% 

0.1% 

MD  >  PCNP 

No.  of  visits  for  other  than  "EKG  condition” 

n-2581 

n*5573 

n**1343 

34 

No.  of  x-ravs  not  for  "x-ray  condition" 

49% 

42% 

41% 

MD  >  PA 

Totai  no.  of  x-rays 

n=372 

n=881 

n*223 

35* 

No.  of  x-ravs  not  for  "x-ray  condition" 

6% 

6% 

6% 

no^ 

No.  of  visits  for  other  than  "x-ray  condition  ' 

n*3045 

n*5697 

n*1497 

36* 

Backache  alone  or  with  sciatica  (only,  first  visit); 

3% 

13% 

43% 

PCNP  >  MD 

x-ray  ordered 

n*29 

n=75 

n*28 

37 

Backache,  no  sciatica  (only); 

9% 

14% 

26% 

no 

x-ray  ordered 

n=44 

n=156 

n=47 

38^ 

Bronchitis  (only,  first  visit); 

31% 

35% 

31% 

no 

x-ray  ordered 

n»13 

n=73 

n*l6 

39* 

Coryza  (only,  first  visit); 

22% 

50% 

36% 

PA  >  MD* 

throat  culture  ordered 

n*60 

n=302 

n=80 

NOTE:  Because  the  action  is  undesirable,  the  lower  the  percentage,  the  better  the  care. 
^Excludes  physical  examinations. 

^Criterion  used  in  sign  tests  and  all  summary  comparisons. 

Q 

Regression  results  Indicated  significant  differences  among  bases*  See  App.  B. 
^Criterion  used  in  sign  tests  only. 


The  remaining  four  criteria  for  undesirable  diagnostic  actions  are  simple  to  comprehend. 
Two  of  them  deal  with  ordering  x-rays  for  the  diagnosis  of  backache.  Particularly  on  a  first 
visit  for  the  sole  diagnosis  of  backache,  it  should  generally  be  unnecessary  to  obtain  an  x-ray. 
X-rays  were  not  ordered  very  often  by  physicians  or  PAs,  but  they  were  ordered  quite  often 
by  PCNPs.  The  difference  is  significant  when  PCNPs  are  compared  with  physicians. 

The  final  undesirable  diagnostic  action  is  of  particular  interest.  Coryza  means  runny  nose; 
it  is  unlikely  that,  when  runny  nose  was  the  only  diagnosis,  a  throat  culture  need  have  been 
taken.  Although  there  are  certainly  some  exceptions,  we  would  expect  to  see  a  very  low  rate 
of  ordering  of  throat  cultures  in  patients  with  runny  nose.  In  fact,  however,  when  coryza  was 
the  only  diagnosis,  a  substantial  number  of  throat  cultures  were  taken  by  all  three  provider 
groups,  and  PAs  ordered  throat  cultures  significantly  more  often  than  did  physicians.  (All 
providers  at  one  of  the  bases  performed  significantly  better,  and  providers  at  a  second  base 
performed  significantly  worse.) 


UNDESIRABLE  THERAPEUTIC  ACTIONS 

"Undesirable  therapeutic  actions”  (Table  C.4)  is  a  class  of  criteria  rarely  discussed  in  the 
quality-of-care  literature.  Once  again,  because  the  action  is  undesirable,  a  lower  percentage 
implies  better  care.  Thus,  the  criterion  concerning  oral  or  injected  steroid  for  asthma  or 
dermatitis  shows  gratifyingly  low  percentage  compliance  by  the  three  provider  groups.  Of 
course,  there  are  some  patients  with  either  asthma  or  dermatitis  who  do  require  oral  or 
injected  steroids;  but  they  should  be  the  exception,  as  our  data  confirm  for  Air  Force  medical 
clinics. 

Was  an  antibiotic  prescribed  when  the  provider  checked  the  diagnosis  of  coryza  (runny 
nose)  or  febrile  cold  (cold  with  fever)?  As  is  appropriate,  antibiotic  was  not  ordered  very  often 
by  any  of  the  groups,  but  it  was  ordered  significantly  more  often  by  the  physicians.4 

Similarly,  for  viral  syndrome  with  or  without  gastroenteritis,  an  antibiotic  was  not  ordered 
very  often,  but  for  viral  syndrome  with  gastroenteritis,  an  antibiotic  was  prescribed  signifi¬ 
cantly  more  often  by  PCNPs. 

There  is  minimal  evidence  of  poor  medication-prescribing  habits  for  obesity,  nor  is  tetracy¬ 
cline  being  incorrectly  prescribed  to  young  children. 

The  "steroid  groups”  can  be  found  at  the  end  of  the  table,  and  the  fractions  serving  as 
criteria  are  presented  at  this  point.  They  show  that  their  use  may  present  some  problem. 
However,  for  only  one  of  these  criteria  is  there  a  statistically  significant  difference  in  the 
prescribing  behavior  of  the  providers. 

Further  evidence  of  generally  good  prescription  practices  with  antibiotics  is  found  in  the 
last  two  criteria  listed  in  this  criterion  class. 


DESIRABLE  DISPOSITION  ACTIONS 

Desirable  disposition  actions  (Table  C.5)  constitute  our  final  group  of  criteria.  We  selected 
a  group  of  conditions  usually  requiring  follow-up  (for  example,  hypertension,  epilepsy,  dia¬ 
betes),  and  asked  for  an  indication  that  some  kind  of  return  appointment  was  planned.  Here, 
there  is  a  statistically  significant  difference  in  favor  of  the  physicians. 

*Here  is  an  area  where  the  ideal  of  zero  percent  compliance  is  unrealistic,  because  there  is  often  great  pressure  from  the 
patient  to  ha  ve  an  antibiotic  prescribed.  Further,  there  are  some  people  with  conditions  such  as  chronic  lung  disease  for  whom 
it  may  be  wise  to  prescribe  antibiotics  in  these  circumstances.  Still,  this  criterion  is  sound  as  a  reflection  of  quality  of  care, 
because  medical  indications  for  using  antibiotics  in  this  situation  are  few. 
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Table  C.4 


Undesirable  Therapeutic  Actions 


HD 

PA 

pci;p 

Statistically 

Criterion 

Only 

Only 

only 

Significant? 

4(H 

Aschma  or  dermatitis;  steroid  (oral  or 

42 

2% 

3% 

no 

injected)  prescribed 

n-143 

n-28i 

n-66 

41-* 

Coryza  or  febrile  cold; 

lb% 

10% 

in 

MD  >  PA‘ 

antibiotic  prescribed 

n-293 

ii“  706 

n=282 

4?* 

Viral  syndrome  without  gastroenteritis; 

14% 

12% 

15% 

no 

antibiotic  prescribed 

n-14I 

n-198 

n-59 

4T! 

Viral  syndrome  with  gastroenteritis; 

12 

22 

92 

PCNP  >  MD 

antibiotic  prescribed 

11“  80 

n- 164 

n-58 

44 

Obesity  (only);  any 

9% 

5  Z 

202 

no 

medication  prescribed 

n«ll 

n-21 

n-3 

45 

Obesity  (only);  tranquilizer 

U% 

0% 

0% 

no 

or  steroid  prescribed 

n-11 

n-21 

n-5 

46tf 

Obesity  (only);  tranquilizer  or  steroid 

0% 

0% 

0% 

no 

or  thyroid  medication  prescribed 

n-11 

n-21 

n-5 

47* 

Tetracycline  prescribed; 

27. 

2% 

2% 

no 

child  age  6  or  younger 

n-58 

n-270 

n-4  3 

4S1' 

No.  of  times  "steroid  condition"  was  not  present 

71% 

b52 

602 

no 

No.  of  times  oral  or  injected  steroid  was  given 

n-28 

n-2  3 

n-5 

49 

No.  of  tiroes  "steroid  condition"  was  not  present 

41% 

25% 

24% 

MD  >  PA 

No,  of  times  any  steroids  prescribed 

n-lOb 

n*  JO  5 

n-50 

50 

No.  of  times  "steroid-contraindicated  condition"  was  present 

7% 

9% 

02 

no 

No.  of  times  oral  or  injected  steroid  was  given 

n-28 

n-23 

n-5 

51<3 

No.  of  times  oral  or  injected  steroid  was  prescribed 

0.4% 

0.2% 

0% 

no 

No.  of  visits  for  "steroid-contraindicated  conditions" 

n-554 

n-9bl 

n-228 

52^ 

No.  of  tiroes  physical  therapy  ordered  w/o  "l>]  condition" 

40% 

20% 

7% 

no 

Total  no.  of  times  physical  therapy  ordered 

n-10 

n-124 

n-3(J 

53 

Osteoarthritis  (only);  steroids  or 

29% 

7% 

0% 

no 

narcotics  prescribed 

n-7 

n-6ft 

n-2 

54° 

Allergies  (only);  antibiotic  prescribed 

1% 

■:l% 

2% 

no 

n-88 

n-2  30 

n-5  7 

55« 

Headache  (only);  antibiotic  prescribed 

22 

12 

0- 

no 

n=4b 

n-102 

n-24 

NOTE:  Because  the  action  is  ^desirable,  the  l.wr  the  percentage, 
the  ret  Mr  the  care. 

aCriterion  used  in  sign  tests  and  all  sunmary  comparisons. 
^Regression  results  indicated  significant  differences  among 
bases.  See  App,  B. 

Criterion  used  in  sign  testa  only. 
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If  a  tranquilizer  or  antidepressant  was  prescribed  on  this  visit,  was  some  kind  of  follow-up 
planned,  as  we  believe  should  have  been  the  case?  For  this  criterion,  physicians  were  again 
acting  more  appropriately,  but  the  difference  is  not  statistically  significant.  In  the  case  of 
planning  a  definite  follow-up  visit  for  infectious  otitis  media,  PAs  outperformed  physicians. 


Table  C.5 

Desirable  Diagnostic  Actions 


Criterion 

MD 

Only 

PA 

Only 

PCiJP 

Only 

Statistically 

Significant? 

56a 

Condition  judged  ’’very  serious/ 
serious"  by  provider;  definite 
follow-up  planned 

81% 

n-374 

85% 

n-245 

57% 

n=74 

MD  >  PCNPfc 

57a 

Condition  that  should  usually 

73% 

66% 

65% 

MD  >  PA 

be  followed  up  (reduced  list); 
definite  return  appointment 
planned 

n-673 

n-778 

n-159 

MD  >  PCNP 

58 

Condition  that  should  usually 

69% 

63% 

60% 

MD  >  PA 

be  followed  up  (expanded  list); 
definite  follow-up  planned 

n-935 

n-1501 

n«=345 

MD  >  PCNP 

59° 

Anxiety  or  depressive  neurosis; 
definite  follow-up  planned 

59% 

n-71 

48% 

n*60 

71% 

n*14 

no* 

Tranquilizer  or  antidepressant 
prescribed;  definite  follow-up 
planned 

54% 

n»143 

49% 

n-115 

38% 

n-24 

nofc 

bld 

Otitis  media  (infectious) (only, 
first  visit);  definite  follow¬ 
up  planned 

63% 

n«4U 

85% 

n-61 

67% 

n-12 

PA  >  MD 

62 

Otitis  media,  either  type 
(only,  first  visit);  definite 
follow-up  planned 

64% 

n»44 

83% 

n*64 

69% 

n-13 

no 

^Criterion  used  in  sign  tests  and  all  summary  comparisons. 


Regression  results  indicated  significant  differences  among  bases. 
See  App.  B. 
c 

Criterion  used  in  sign  tests  only. 

d 

Criterion  used  in  sign  tests  and  PA-MD  comparisons,  but  not  in 
PCwP-MD  comparisons. 


Table  C.6 

Diagnostic  Groups 


STEROIDS  (.ORAL  OR  INJECTED) 

Acceptable  group 

Asthma 

Dermatitis  (including  eczema) 
Malignant  neoplasm 
Rheumatoid  arthritis 
Other  arthritis /rheumatism 
Burs it is / t enosynovi t is / synovitis 
Other  diseases  of  intestine  and 
peritoneum 

Contraindicated  group 

Ulcer  disease 
Diabetes  mellitus 
Hepatitis 
Obesity 
Hypertension 

Functional  upper  G1  distress 
Functional  large  bowel  distress 
Backache  alone 
Backache  with  sciatica 

FOLLOW-UP  DESIRABLE  GROUP 
Hypertension 
Alcohol  abuse 
Diabetes  mellitus 
Psychosis 

Anxiety  or  depressive  neurosis 

Drug  dependence 

Epilepsy 

Ischemic  heart  diseases 
Chronic  obstructive  pulmonary 
disease  (COPD) 

Rheumatoid  arthritis 
Poisoning,  overdose 
Malignant  neoplasm 

Expanded  list  also  includes; 

Infectious  mononucleosis 
Iron  deficiency  or  other  anemia 
Sexual  dysfunction 
Adult  situation  disturbance 
Problems  of  development, 
retardation,  or  behavior 
Otitis  media  (serous  and  nonserous) 
Pneumonia 
Asthma 

Disorders  of  menstruation 


ERG  GROUP 

Thyroid  disease 

Diabetes  mellitus 

Ischemic  heart  diseases 

Arrhythmias  or  heart  block 

Other  heart  diseases 

Hypertension 

Syncope 

Cholecystitis 

Heart  murmur 

X-RAY  GROUP 
Asthma 

Tension  headache 
Ischemic  heart  diseases 
Arrhythmias  or  heart  block 
Other  heart  diseases 
Acute  sinusitis 
Pneumonia,  pneumonitis 
Acute  bronchitis 

Chronic  bronchitis,  emphysema,  COPD 

Chronic  sinusitis 

Cholecystitis 

Osteoarthritis 

Rheumatoid  arthritis 

Other  arthritis /rheumatism 

Bursitis,  tenosynovitis,  synovitis 

Backache  with  sciatica 

Backache  alone 

Pain  in  joint 

Other  headache 

Fracture  of  lower  limb 

Fracture  of  upper  limb 

Dislocation,  upper  extremity 

Other  fracture 

Sprain/strain  upper  limb 

Sprain/strain  lower  limb 

Sprain/strain  neck/back 

Heart  murmur 

Dislocation,  lower  extremity 
Trauma  to  head 
Muscle  pain 

Other  bones,  joints,  muscles 
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Table  C.6— continued 


PHYSICAL  THLRAPY  GROUP 

Osteoarthritis 
Rheumatoid  arthritis 
Other  arthritis /rheumatism 
Bursitis/tenosynovit is /synovitis 
Backache  with  sciatica 
Backache  alone 
Pain  in  joint 
Fracture  of  lower  limb 
Fracture  of  upper  limb 
Dislocation,  upper  extremity 
Dislocation,  lower  extremity 
Other  fracture 
Sprain/strain  upper  limb 
Sprain/strain  lower  limb 
Sprain/strain  neck/back 
Other  bones,  joints,  muscles 


LUNG  DISEASES 
Asthma 

Acute  bronchitis 
Pneumonia 

Chronic  lung  disease 

PELVIC  EXAMINATION  INDICATED  CROUP 

Vulvitis,  vaginitis,  or  cervicitis 
Menopausal  symptoms 
Disorders  of  menstruation 
Dysfunctional  uterine  bleeding 

HEART  DISEASES 

Ischemic  heart  disease 
Arrhythmias  or  heart  block 
Heart  murmur 
Other  heart  diseases 
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Appendix  D 


RELATIONSHIP  BETWEEN  UTILIZATION 
AND  QUALITY:  LITERATURE  REVIEW 


The  question  of  the  relationship  between  the  use  of  laboratory  tests  and  the  quality  of  care 
is  a  difficult  one  to  approach.  The  work  of  several  groups  exemplifies  the  best  efforts  in  this 
area. 

Schroeder,  Schliftman,  and  Piemme  (1974)  examined  the  relationship  between  variation 
in  use  of  laboratory  tests  and  the  quality  of  physician  performance.  The  physicians  studied 
were  21  medical  interns  in  a  major  teaching  hospital.  The  rank  order  correlation  between  the 
cost  of  an  intern’s  treatment  regimen  and  his  assessed  clinical  competence  was  negligible. 
Daniels  and  Schroeder  (1977)  compared  variations  in  laboratory  use  with  measures  of  clinical 
productivity  and  outcome  of  care,  for  ambulatory  patients  carrying  the  diagnosis  of  hyperten¬ 
sion.  Although  there  was  great  variation  in  mean  annual  laboratory  costs  per  patient  among 
a  group  of  faculty  internists,  there  was  no  positive  association  between  a  physician’s  frequency 
of  laboratory  use  and  either  clinical  productivity  or  outcomes  of  care. 

At  the  University  of  Utah,  Wright,  Kane,  Snell,  and  Woolley  (1977)  examined  the  relation¬ 
ship  between  levels  of  medical  training  and  direct  costs  including  fees  for  provider  services, 
for  a  large  number  of  episodes  of  acute  illnesses  treated  in  ambulatory  care  clinics  of  the 
Family  Practice  Residency  Program.  Outcomes  of  these  episodes  were  also  examined.  Al¬ 
though  average  total  cost  per  episode  was  not  related  to  provider  type,  there  were  significant 
differences  among  providers  in  laboratory  and  medication  costs.  Physician’s  assistants  (PAs) 
showed  the  highest  average  total  costs,  although  the  difference  was  not  statistically  significant 
from  the  costs  for  faculty  members  or  residents.  Furthermore,  as  the  authors  point  out, 
"although  PAs  generated  higher  costs,  especially  for  patients  with  bad  outcomes,  these  figures 
may  be  inflated  because  the  services  of  all  providers  were  charged  for  equally.  Also,  PAs  were 
closer  to  the  faculty’s  costs  than  were  any  of  the  residents.  This  may  reflect  a  closer  superviso¬ 
ry  relationship  than  that  usually  found  between  faculty  and  residents...  .’’Faculty  members 
had  the  highest  laboratory  costs,  PAs  were  second,  and  second-year  residents  had  the  lowest. 
One  unexpected  result  of  the  study  was  that  "the  PA  achieved  more  good  outcomes  than  either 
the  faculty  or  any  of  the  residents,  although  they  saw  patients  of  the  same  type  and  degree 
of  severity.”  The  authors  raised  the  possibility  that  this  higher  percentage  of  good  outcomes 
"might  be  considered  an  indicator  of  the  value  of  more  active  supervision,”  and  pointed  out 
that  their  data  suggest  that  "such  increased  supervision  does  not  necessarily  increase  the  cost 
of  care  for  patients  with  good  outcomes.” 

It  should  be  recalled  that  this  study  was  limited  to  acute  conditions,  which  comprise  only 
one  segment  of  the  totality  of  problems  encountered  in  the  ambulatory  care  setting.  For 
laboratory  services,  the  PA  ranked  third  most  expensive  in  use  of  laboratory  tests  for  those 
episodes  showing  good  outcomes  (87  percent  of  episodes  in  the  case  of  PAs),  and  most  expen¬ 
sive  in  laboratory  utilization  for  those  episodes  that  resulted  in  a  bad  outcome.  In  summary, 
this  study  did  not  provide  evidence  of  excessive  laboratory  use  by  PAs.  In  the  overwhelming 
number  of  cases,  their  utilization  of  laboratory  tests  (as  measured  by  the  costs  of  these  tests) 
was  in  the  same  range  as  that  of  physicians;  and  in  the  small  percentage  of  cases  where  a  bad 
outcome  occurred,  their  use  of  laboratory  tests  was  high,  but  virtually  identical  to  that  of  their 
faculty  supervisors. 

Finally,  the  work  of  Record,  Hurtado,  and  O’Bannon  (1977)  deserves  mention.  Using  chart 
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abstraction  techniques,  they  gathered  data  on  the  use  of  laboratory  and  x-ray  tests  by  five  PAs 
and  53  physicians  working  in  outpatient  settings  of  the  Department  of  Medicine  at  Kaiser- 
Permanente,  Portland,  Oregon.  They  chose  four  morbid  conditions:  strep  throat  (note  the 
specificity  of  the  diagnosis,  rather  than  the  more  general  pharyngitis),  coryza/upper  res¬ 
piratory  infection,  bursitis,  and  bronchitis.  For  strep  throat,  PAs  ordered  throat  cultures  for 
98  percent  of  patients  as  compared  with  physicians’  84  percent,  and  PAs  ordered  x-rays  for 
5  percent  as  compared  with  physicians’  1  percent.  For  upper  respiratory  infection,  PAs  or¬ 
dered  throat  cultures  for  41  percent  of  patients  as  compared  with  physicians’  31  percent,  and 
PAs  ordered  x-rays  for  7  percent  as  compared  with  physicians’  6  percent.  For  bursitis,  the 
picture  is  somewhat  different:  Both  provider  groups  ordered  x-rays  for  20  percent  of  patients, 
while  PAs  ordered  additional  related  laboratory  work  for  5  percent  as  compared  with  physi¬ 
cians’  18  percent.  Differences  were  most  striking  for  bronchitis,  where  PAs  ordered  x-rays  for 
59  percent  as  compared  with  physicians’  31  percent;  PAs  ordered  throat  cultures  for  24  percent 
as  compared  with  physicians’  10  percent;  and  PAs  ordered  any  other  laboratory  test  for  52 
percent  as  compared  with  physicians’  16  percent. 

The  authors  state  that  "figures  concerning  use  of  lab  and  x-ray  services  for  these  particular 
morbidities  suggest  that  the  PAs  tend  to  practice  more  conservatively;  that  is,  they  rely 
somewhat  more  heavily  upon  supportive  diagnostic  services,  especially  lab  tests.”  However, 
"comparison  of  MDs  and  PAs  with  respect  to  use  of  supportive  diagnostic  services  over  a  wider 
range  of  morbidities  showed  little  difference  between  the  average  MD  and  the  average  PA” 
(emphasis  added).  Working  with  the  same  data,  a  medical  review  team  concluded  that  "the 
findings  strongly  suggest  that  within  the  stated  frame  of  reference,  PA  performance  compares 
quite  favorably  with  that  of  physicians. . . .  Certainly  there  is  no  evidence  that  PAs  provide 
inferior  services. ...  PA  performance  compares  well  with  MD  performance  within  the  range 
of  the  non-complex,  routine  cases  which  define  PA  practice  at  Kaiser-Permanente,” 

Thus,  although  no  direct  connection  is  drawn  between  utilization  and  quality,  the  juxtapo¬ 
sition  of  data  and  comments  in  their  report  reveal  that  the  relationship  was  very  much  on  the 
authors’  minds.  In  general,  they  found  "little  difference”  in  utilization  of  tests  between  PAs 
and  MDs  (although  the  specific  data  they  present  clearly  show  heavier  reliance  on  tests  by 
PAs),  and  they  were  satisfied  with  the  quality  of  care  being  delivered  by  PAs.  Their  conclusions 
are  similar  to  ours. 


Appendix  E 

DIAGNOSTIC  CATEGORIES 


This  appendix  lists  the  conditions  that  make  up  the  categories  used  in  Sec.  V.  The  category 
"physician  usually  not  needed”  includes  all  conditions  on  the  Patient  Contact  Record  not  listed 
here.  (Such  conditions  as  "other  nervous  system  diseases”  refer  to  categories  of  conditions 
found  on  the  Contact  Record.) 

Physician  Usually  Needed  on  First  Visit  for: 

Malignant  neoplasms 
Psychosis 

Cholelithiasis,  cholecystitis 

Other  nervous  system  diseases 

Arrhythmias  or  heart  block 

Problem  of  development,  retardation,  or  behavior 

Heart  murmur 

Other  heart  diseases 


Physician  May  be  Needec*  on  First  Visit  for: 

Vascular  lesions  (includes  stroke,  cerebral  arteriosclerosis) 

Thyroid  disease 

Cervical  erosion 

Fracture  of  lower  limb 

Fracture  of  upper  limb 

Fever  of  unknown  origin 

Disorders  of  menstruation,  dysfunctional  uterine  bleeding 
Diabetes  mellitus 
Ischemic  heart  diseases 

Breast  mass  or  breast  disease  (excluding  malignancy) 

Drug  dependence /abuse 

Rheumatoid  arthritis 

Epilepsy,  convulsions 

Poisoning,  overdose 

Pneumonia,  pneumonitis 

Hypertension  (HBP) 

Anxiety  or  depressive  neurosis 
Other  eye  diseases 

Other  diseases  of  the  esophagus,  stomach,  duodenum 
Other  diseases  of  intestine  and  peritoneum 
Abdominal  pain  (not  otherwise  specified) 

No  definite  diagnosis  at  this  time 
Other  diseases  of  the  reproductive  system 
Menopausal  symptoms 


73 


Other  local  infections  of  skin  and  subcutaneous  tissue 
Diseases  of  sweat  and  sebaceous  glands  (including  sebaceous  cyst) 
Migraine  headache  (or  other  migraine  manifestations) 

Syncope 

Ulcer  disease  (stomach  or  duodenum) 

Trauma  to  head 

Chest  pain  (unknown  origin) 

Asthma 

"Other” 

Family  planning/contraception/  infertility 
Other  anemias 

Cellulitis  (including  lymphangitis) 


Physician  Usually  Needed  on  Return  Visit  for: 

Malignant  neoplasm 
Psychosis 

Other  nervous  system  diseases 
Arrhythmias  or  heart  block 


Physician  May  be  Needed  on  Return  Visit  for: 

Thyroid  disease 
Diabetes  mellitus 
Drug  dependence/ abuse 

Migraine  headache  (or  other  migraine  manifestations) 
Epilepsy,  convulsions 

Problem  of  development,  retardation,  or  behavior 
Other  eye  diseases 

Ischemic  heart  diseases  (including  angina  pectoris,  ASHD) 
Heart  murmur 
Hypertension  (HBP) 

Other  diseases  of  esophagus,  stomach,  duodenum 

Cholelithiasis,  cholecystitis 

Other  diseases  of  intestine  and  peritoneum 

Disorders  of  menstruation,  dysfunctional  uterine  bleeding 

Menopausal  symptoms 

Cervical  erosion 

Breast  mass  or  breast  disease  (excluding  malignancy) 

Rheumatoid  arthritis 

Trauma  to  head 

Fracture  of  upper  limb 

Fracture  of  lower  limb 

Poisoning,  overdose 

Fever  of  unknown  origin 

Chest  pain  (unknown  etiology! 

No  definite  diagnosis  at  this  time 
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